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ABSTRACT 

Full teits are provided of the individual papers 
presented at a symposium held to investigate technology "based systems 
for improving productivity in higher education. These systems, which 
marry an engineering systems approach with behavioral objectivism, 
are studied in the context of three specific learning environments: 
1} grouped and bounded environments, 2) individualized bounded 
environments, and 3) personalized open environments. Each paper 
discusses a technology-based system supportive of one of these three 
learning environments and deals with the topics of mode and media, 
management, and measurement as addressed by their systems. Comments 
of discussants and reactants are also included; these analyze the 
effectiveness of the systems in supporting learning objectives, their 
ability to measure outcomes and manage resources, their capacity for 
gaining acceptance and validation, and their relationship to other 
demonstrated systems operating in common learning environments. 
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PREFACE 



Symposium 

The United States Office of Education drui the Nationai institute of Education ^intiy sptmsored a three- 
day Symposium, in September of 1973, on technology -based systems for improving productivity in Higher 
Education. The Symjjosium was hostetJ by the State University of New York at Stony Brook under the joint 
$U(>ervision of Professor Sheltey A. Harrison of the School of Engineering and Professor Lawrence M. Stolurow. 
Chairman of the Department of Education. 

Htgher Education faces mcreasing demands for more and varied educationai experience by the consumer 
(students and parents) and diminishing fiscal resources available to su|;^3ort this educationai enterprise. Tech- 
nological systems offer the potential for meeting a number of these needs including an increase in prcKiuctivity- 
This Symposium examines a rar>ge of critical issues relating to the $uc(%^ful use of educational tec^noiogy 
directed toward increasing productivity in both traditional and nonttaditionai settings. Equal conoern is given 
both fiscal and quality of life variables as are gains in cognitive skills, critical thinking, and improved attitudes of 
all involve^J with education: students, faculty, administrators, parents, 3nd community. 

Technology t)ased educational systems are best studied in the context of specific fining environm&its. 
This approach places primary emphasis on real learning needs and sets the stage for discussing potential tech- 
nology-based solutions. The essential premise of this Symposium is that a thoughtful marriage of an engineering 
"systems approach'' and behavioral objectivism may well offer the support necessary for wider access to better, 
more mdtviduali/eil learning at improved costs. 

In Institutions of Higher Educatioii and Institutions of Continuing Education, today's society is con- 
fronted with three learning environments: 

1 . Grou^wd and Bouniied Learning Environments 

2. Individualized, Bounded Learning Environments 

3. Personalized, Open Learning Environments 

Each of these three learning environments is, in turn, a day-theme for our three-day Symposium. Space- 
time 6o4/m:/^ environments refer to institutionalized educational systems exhibiting definite learnir^ require- 
ments, entrance requirements, physical locations, and definite space, time, and resource constraints on the 
teaching/learning process. Technology support of both conventional and individualized learning in bounded environ- 
ments occupies the discussions on day-one, and day-two, respectively. The third day ^dresses open, more jMrsonal- 
imi environments free of manv requirements and constraints of institutionalized education. These include home 
study, continuinq education, and cultural activities. 

Each exptTt $f^>e3ker addresses a technology based system supportive of a specific learning environment- 
Speakers discuss mode and media, management, ami measurement as addressed by their systems. 

Mode and Media: What pr oc:e<lures are utilized to define educational goals and institutional 
ohi«.>rtivps? How are mode and media treatments decided upon? What are major considera- 
tffjos tn production, deliverv, maitUonance, and revision of system components? 

VI 



Management: VWio takes part in the learnii^ environment? How does personal manac^ment 
procmJ? What form does pwsonal mana^ent take? VWiat form does prc^am and Institutional 
management take? How are resources (people, materials, space, time) managed? 

Measurem&it: How does the system provide for measurement and ad^ive feedback on 
personal achievement, effectiveness of environment (pec^e. things), learning (cognitive, skill, 
affective), costs (capital, develofwnent, reajrrent, revision)? 

Discussants and reactants at each session anatv2e: 

, , , effectiveness of system in sup?K>rtfng the learning objectives 

. . . effectiveness of system in measurement and mana^ment of resourw aH<K:ation and costs 

. . , effectiveness of system in gaining accept^ce, valkJation, and providing adequate teacher training 

. . comparative relationship to other demonstrated systems operating in common learning environments 

Early in the Symposium it became obvious that the issue is not whether educational technology can w^ < and 
mdeed improve productivity but whether institutions of higher education are, in fact, prepared to use it. Clearly, 
in order for educational technologies to be used effectively, an institution must have a well-defined sense of 
purpose, direction, and commitment. Campuses where tei^nology has been and continues to be successful surely 
prove this point. 

It is oi»r hope that the information in this report will lead to the organization of study groups on campuses 
which will develop plans for action and policies to foster the growth and development of the most suitable and 
cost-effective technologies to meet local needs. 

Productivity 

"There is little confusion about the definition of productivity; outputs of a process are related to inputs and 
the relationship is taken to indicate- productivity. The diffioilties arise in making the concept operational and 
applying it."^ 

Unlike the commercial sector where profitability provides a pervading production criterion, the higher 
Education enterprise as yet possesses no useful set of measuremem and interpretation pra^dures. A theory of 
teaching is long overdue. Without it we can only continue to be unsatisfymgly descriptive in our use of intrinsically 
limited productivity ratios such as FTE's. cost per student hour, etc. 

Bound up in these questions of output/input measures of effii:iency is the perhaps more fundamental 
question of effectiveness. But this requires the precise definition of i istructional goals and learning objectives 
bearing in mmd relevance to leaners' present and future life and job environments. The production of intelligent, 
capable citizens is the charge and responsibility of institutions of higher education. Until the ^als (benefits) of ' 
this process are clearly enunciated we cannot even begin to develop useful cost/benefit analysis. Productivity 
analysis should be a dynamic, evolutionary process of setting educational goals, continually estimating and 
monitoring goal production costs, and improving production and output for the changing inputs. 

Application of technology oDncepts and techniques to the general problem of resource allocfjtion and 
program plannmg and budgeting is still in its infancy. This Symposium treats these matters in only rather general 
terms placing its major emphasis on the instructional process and suf '>ortive technology-based development 
and presentation systems. 



'Producffvitv Burden of Success, by Wilham ToornUs, £RlC/Highef Fducotioi Pwarch Report. No. 7, 1973. 
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Editcatfofidt Tectifioiogy 



The Symposium addre^es mainly the instructional aspects of educational techm>ii^tes as these are of c^tral 
priority and have receimi the maforitv of attention. r^:iinology promotes a tnm systen^ approach in the ad^^tive 
development of curriculum, ir^trunionai delivery, performance, n^surement^ arKl evaluation. Course cuiricuta 
are defined behavioraity in terms of their cognitive and affective ^struction objectives-an imperati^ pr^ess 
enabling unambiguoia communication and thus ei^ancing transferability of materials and rneasi^abiltty of 
performance. Instructional objecti^s are organised into small logical sets or modules amenable to instructional 
delivery sufH>ort in one or another i^^iiNrt^riate mode ami medium combination. The speakers of this Symposium 
investigate the many ways in vvhich te<^nokigy supports the numerous strategies of instructitKi f the group 
paced throu^ the various self-paced, individualired environments that atxonKnodate natural var^tions in 
learner preparedness, aptitiKle, and interest. The talks are by no means exhaustive in tfieir coverage (that being 
well beyond the intended sco^ of the Symposium). They do, however, treat in depth most of the major 
successful technology endeavors in instructional learner-teache*' organization and ddivery in Higher Education 
today. The report reveals the status of several promising experiments currently in progress around the nation, 
as well as, extensively documenting the development and delivery successes of England's Open University. 

Ittues and Probien^ 

Open and frank discou se over a full spectrum of educational issues and problem areas pervaded the ^-essions 
and numerous informal gatherings. Discussions ymre vigorous and often argumentative. The air was one of deep 
concern for the issues ami noticeable dedication to improving the quality of learning and reducing the cost of 
instructional delivery. Below, are hi^lighted several of the ar^s of major concern: 

• Lack of adequately researched models of learning sufficient for higher education. 

• inadequate models and procedures for (a) curriculum and materials development, (b) formative evalua- 
tiofi, <c) faculty training, and (d) productivity analysis. 

• Existence of major traditional foci of resistarM:e to instructional innovation and development within 
institutions of higher education. 

• Lack of legitimacy afforded research m learning and instructional system development. 

• Inaiiequate fKofessional and financial incentives. 

• Lack of coordinated methods and agencies for trans^rr and dissemination of R & D results in education- 
al technology projects and applications. 



The State University of New York campus at &ony Brook is ^ppy to have been $ele<iied as the site for this 
forum on Higher Education. The ^ool of EngiTOerlng and Itepartmrnt of Education at Stimy Brook are actively 
engaf^d in numerous experimental and dewtopmentat proiects mvofving teclif>oiogy4iased (earning systems. We 
hope that there wilt be free and rapid a>mmuntcation md di^emination of experience and materials in educdti<mal 
tm:hnoiogy efforts ^oss the nation. The nwd anf< opportimity for innovation atd ch^>^ are at hwd. It is our 
common respor^ibiiity to n^ke thoughtful and effective use of availaUe communication tedmoioj^es* 

The editors of this report acknowledge Fern Singer's tireless org^izationd and administrative efforts. 
We aiso thank Mr. Charles Miller foi taping the Symf^um ^ssicHis and editing them onto individual c^ettes, 
and Ms* Newilda LaGrandier for iier tireless assistance in preparing copy and typ^t* 
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TECHNOLOGICAL INNOVATION IN A HOSTILE ENVIRONMENT: 
PROBLEMS OF INCREASING PRODUCTIVITY IN HIGHER EDUCATION 

Keith Lumaien 
Professor of Economk^ 
Graduate Business School, Stanford University 

INTRODUCTION 

This pafKir is conc^ned with the problems of inircKiucing and evatt^ting technologtcai innovations in the 
higher education industry. Given the incentive systems existing wUhin established universities the paper argi^ that 
significant inaeases in educational (»rodi^tvity occuring throt^ tfte adoptic^ of technological innovations will 
be possible only if there is a massive research effort. Sudi a res^rch effort is likely to j^ove invaluable even if it is 
mainly new institutions, unfettered by vested interests, which take advanta^ of the researdi results. The f^per is 
in four parts. In the first, the various fKissfble functions of the university are discussed tc^tfier with the problems 
encountered when differences In f^i^ived functions cause conflict. In the mtxmd part, the n^r evades to 
diange are noted, given the existing structure of most of our major instituttOTis of hi^ier learning. The third portion 
is concerned with the inadequacy of mudh of past researdi the economics of higl^r education. Finally a profX>sed 
researdi scheme is spelled out, whidi does not constitute an anempt to solve all of the problems raisfKi earlier, but 
which is desired to yield the type of information on different pedagt^es required by university decision makers 
wishing to increase productivity in higher education. 

COMPLEXITY CF THE UNIVERSITY 

Economi^^ have not been particularly helpful in providing a theory of the university whidi is both realistic 
and useful for policy implementation. Traditional theories of optimizing behavior, of equating marginal scxjial 
benefits with marginal social costs are theoretically valid but of Uttle practical significant for the university 
administrators. Indeed, many of the economists con^rned with efficiency problems and armed with the tools of 
their trade, have avoided efficiency studies in the higher educational industry since that whidi is to be maximizwt 
cannot be agreed upon by a majority of the divei:se interested parties. 

The functions of the university are several, the interest groups the university serves are many, the output ^ 
choices of the university for all practical purposes are infinite, the prices of many of the output elements are not 
readily available, the underlying engineering production functions have seldom been identified, and, finally, 
current institut<onal arrangements discourage actiois which could lead to less ineffidency. 

While many of the above points are obvious, identification of each may be necessary if universities are to be 
forced to move towards the existing efficiency frontier or if universities will be compelled to adopt technological 
innovations which shift out that frontier. 

Perhaps the most widely held view of the function of the university is that of capital creation. Each year the 
university's intake is a crop of freshmen, eager and anxious to increase their net worth. (It is important to note 
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that with few ei&cepttom, ireJimen ^mut to emer untvemtv are not chosen randomSy by a unii^rsiiy from the 
"oo4te9e age" po^dti<y> or even r^^^mly fiom att apfriying. Does this imply that uti*vefSrti€S bdrei^ that the 
greatest wati^ ^Ided is attaanrtirte by sfmitttrMoi the t>f iglhtest students as nwasured ty scholastic aptitude tests? 
Two mam paths, ^htch may trayet several miles as one. can be followed. The hrst ^ovides the stt^fent «iHth a 
general education; he does not team any n^ieciiic skiHs but h«s ^tafyiicaf abiittu s and sensibitities are shaiperied; 
his fiotentidf prodi^ivity b ei^anced and he may utihae feity resources m the future finding out where his true 
oocupatlonai interests lie. The other path ts for the student n^o fcnoi%s« or (earns ciuickly, where he is going. He 
wanis to buitd bridge and the engir^aenng and tedtncrfogy faculties show him hoKV to build them, the social 
sciences teach him {or shou!d teadi him} «^iere^ when and if to buifd them. But the Qi^tion must then arise^ 
tto we need the institution of the university^ we now I ' . it» to achieve these atms. Why not utilize vicfeof ^pes, 
corresponcknoe materials, etc., <^ test the student by national examination, why must we construcc^expereive 
butkfmgs and brif^ students together for sew?rai v^^rs on a Ci s*ipus? From tfw point of visrw of tf*e university as a 
producer of hun^ capital, the answer must tie that sturfents learn enouc^ f lom each uther by tieing together or in 
dose contact with the faculty to justify such costs. If we modify the Dure cap»fal formation jpproadi and allow 
consuntption Uenehts to t>e «kied« a case can tie made for ttte campus university i* these cor«umptK>n t>enefits 
Iplus the po^ible mvestn^nt bewfits mentioned above} Qutv%e«di the co^u. 

A second function at the university h to provide to prospective employers -nformation ^>out jcib candidates 
of different abilities who attend ttie untvefsity mi to these foh candidates information about their refative ^ilities 
to perform different tasks. Orw extreme hypothesis asserts that a untversity education adds nothing whatsoever to 
the abilities or capacities of an individual, but provides a set of filters and a set of obstacle courses through which 
aru! over which an lOdii^idual m4jst pass if he wishes to receive tf>e official stamp thdt tip is ably suited for society's 
better jobs. While, in this view, no inaease in any o^'e irwlividua^^'s productive ca^)acity occurs throu^ university 
attendance and graduation, the whole process by providm^ useful mformation to employers and graduates is 
extremely valuable to them. Given that the filter is effective, an employer no longer liears the risk of hiring 
unsuitable job candidates and can more effiaently allocate different types of latMr to different fob categories. 
CXitput is increased, employers are better off and support the filtering institution as do studt^nts who expect to 
gr^uate. 

Treating the university as a filter yields some stanlir>g results. First, smce more able individuals lienefit by 
being officially classified as superior, even tf no consumption or investment benefits iiccrmed from attending a 
university, they would be willing to pay for an "education" if the expected lifetime earning, appropriately 
discounted, exceeded the real costs. One mitcome. m this circumstance, at odds with the traditional assumption 
about higher education, is that the private retur*- from higher education may exceed ifie social return, i.e., the 
return to graduating students can exceed the productivity increase resulting from iheir "education", the differ- 
ence being a decrease in incomes to the unftltered work&^s* 

Second, if the filtering system were ineffective, i.e.. if unsuitable |Oh candidates managed to be admitted to 
and graduated from universities (for example, through zero entrance requirements and watered dmvn cour^ and 
examination stas>dards), the university would no longer serve any useful filtering function, everyone having ^ 
degree prevents a university education f rcmi being a distinguiiyiing employee «^i5ractertstic. 

Third, if the chief filter is betnqadmitted to the university, i,e., everyone admitieil eventually graduates, 
then the university as it exists should be abohstied with the exception of the admissions committee and the 
admitting office. Wten admitted, stixients are awarded a degree and the empioyer has the information ht' requires 
for efficient labor allocation among different jobs. 

A third function of the unr.^sity, claimed «n tioth word *»nd dewi by tTi^ny Idtulty mfmliers to he the 
mam function of the university, is the production of research and rt?pt>siiory (if infdi n>atiun «ind data. How much 
research and which research should be undertaken are not easily answereii imiujI iuyny the vawiKunt of 
maximizing society's welfare. What is agreed upon is the fact that most oi tht^ r fse*.. ch output «s a public *|oo<l 
dfuJ can tit* liseif by a^y ind^vifliMl in socjetv *v^t^f'KJt o^t^ut^" ^ tV^i-^t. t' Mv mi* r * -hr fd^m^ ttut <i<rtat<*'i 
that iX be unancefi out of oubhc funds, <f indeed it rs wofth pKHkit 



The university not only produce its own research output but srfso stores that output and copies of the out- 
put of similar mstitutions in libraries and computer centers. In addition, information and knowledge is embodied 
tn the academic human capital within the university, i,e., in the community of acaciemic S(^o(ars. This stock of 
capital, both non-human and human, that is stored in university is avatiabie to pass on knowled^ to future 
generations (edLcation) and to nwke further additiom to safety's fund of knc^iedge (researdiK 

One could hypothesize other functions of a university, such as serving iooai or national pride or iM'oviding 
a "baby -sitting*^, perhaps more accurately ''adolescent-sitting'', service to parents and so forth, but such functions 
are less seriously claimed than the three discussed above. The tmportani^ of being aware of the several possible 
functions of the university is thai many actions of individu^ or groups a^ociated with the imiversity are directly 
related to what they li^rceive the function of this institution to be and very often differeni^ tn th^ perceptions 
lead to significant conflict within the university. Such conflict poses significant constraints on the ^iHty of the 
university to achiew efficiency in the rfiort-run, i.e., with existing faculty and facilities* But i^rhaps of even 
greatt^ significance, they limit the pace by which technological (^ogress can proceed in the university. 

QbftiM^es to Change 

Within most universities there are five distinct ''inta'est" groups: stuctents, faculty, administration, library 
and computer personnel and other service staff. Some students - especially advanml gr^uate students — become 
part-time faculty members and some faculty members in turn become part-time administrators. For the sake of 
simplicity we shall ignore the internal a>nflicts of those who wear two hats. 

Outside groups with interests in the university include government, foundations and private donors, 
professional assodations, business firms, parents and potential students. 

Essentially, outside interest groups determine the budget constraints within which all modern universities 
must operate. The government and parents finance the bulk of teaching and the government, foundations, private 
donors and businesses support most of the research. The government is urnier political pr^^^ure from households 
with children to provide college and university education at low cost. FourKJations and private donors may be 
concerned with the ^neral v\^lfare of society and believe that money sf^nt in hi^er education furthers this aim. 
CH they may have a narrower objective and specify that their donations be spent on "neglected" research or 
"Critical" teaching programs. The private donor may also be concerntfd about leaving a monument to his or some 
relative's memory. Unfortunately the monument is often more grand and less efficient from the teaching/researd) 
viewpoint than it need be; in this respect what tf^ churdi in bygone days obtained, the university now receives. 
Business firms, tike the government, are concerned with both the graduates of the university ami its research out- 
put and in some instances with the social ima^ att^ed to the supfwrt of "worthwhile" activities. Parents 
undoubtedly differ in their perceived function of the university. Some view it from the value-added side (con- 
sumption, investment or both), others favor the filter hypothesis and still others see the university as the institution 
within which their children t>ecome adults. Finally, prospective students may see the university in any one or in 
some combination of its many possible roles. 

Without t>eiaboring the point there is ample evidence that when resources are not being allocated in the 
university in a manner which meets the approval of an outside group, pressures are often exerted to "rectify" the 
situation. When the outside group approves, appreciation can take the form of increased support, both financial 
and political. The retention of "communist" professors and the build-up of the successful football team are often 
quoted examples. 

Perhaps even greater conflicts of interest He within the university itself. Once admitted, students tend to 
demand (besides subsidized housing, subsidized food, recreational facilities, etc.) "academic power." Agitation 
exists for "relevant" courses, black studies, reduced or no course requirements and the abolition of grading. A 
significant poiiion of administration and faculty believe that students should have no, or few, consumer 
sovereignity nnhts. The students have not beer\ around long enough to appreciate the importance of prerequisites, 



they can be manipulated by the charismatic but incomp^ent fH^ofes^r in the contenttess course^ and once 
admitted they have an incentive to *'water-down" courses and erode grading standards. The adnrHntstrators who 
want to preserve the integrity of the filler and the faculty who are concerned with academic fv^esti^ haw a 
da;ire to maintain high standards. Part of the conflicl arises because of the different perceptions the various 
mterest groups have in the main function of the university. For example, thc^ students who want a degn^ as a 
certification that they have passed through the unhrersity filter undoubt^fy want to maximias the prc^iability 
that they will get a degree and also want their stay at the university to have as hi^i a cof«umption value as 
possible. From this $tand|K)int it is quite rational to oppose any restrictions impwed by the administration or 
faculty* 

VWiether students believe the value added or the filter theory, once a student is admitted to the university 
many ^ds and service are provided, or are exf^cted to be provided, at zero incremental financidi cost to the 
student. Faculty time is one such commodity where conflicts ari^. Since students are not diar^ tuition by 
the amount of faculty time they consume, there js no explicit reason why the student should iwt consume a 
faculty member's time until marginal benefits equal marginal fzero?) costs to the stuctont. The Of^rtunity cost of 
all <most?} faculty men^ers^ time is significantly greater than zero, and every hour spent with e sttKlent is an 
hour less of research, administration or leisure* Since the payoff to a professor in spending nu5re time with a 
student than that ''dictated" by the administration is, to all intents and purposes, zero, professots will not only 
make themselves ^rce but also avoid where (K^ssible the fiats laid dcw/n by the administration. Those faojlty 
who are readily available to students seldom receive the plaudits of their colleagues because they make their 
coiteac^es appear as shirkers and also by |ust being there are liable to inherit their aiisent coltea^fes' ^are of 
problems and students. Sim:e ih&e is no rationing device for determining optimal student-faculty contact, this 
will remain an area of conflict betv\^n students and faculty. The abserrce of incentives to resolve this conflict 
efficieotiy is apparent; in those few institutions where students are given a choice of attending iiw lectures or 
live television (or vid^taped) instruction. Assuming the same quality of instruction, there is little incentive for 
the student to give up the spontaneity and intimacy of the live lecture when there is no difference in cost to him. 
The fact that production costs are substantially different for the two outputs dc^ not affect the student becau^ 
he is charged a flat tuition fee no matter which pedagc^y he elects. 

This is not to argue that technological innovation$ within the existing framework could not benefit enrolled 
students. If, for example, students were shown that the r^ources saved from videotaping say 5 to 6 required 
courses were to be allocated to additional course offerings, small seminars or whatever, the benefits would be 
obvious. If however the choice is between the live versus the non-live l^ure, for example, CBt&is paribus the 
latter will almost consistently prove to be less appealing, especially if it is an isoia^d case. Wiat maki^ soUhd 
economic sense within the ffat tuition fee system of most universities would be a tuition refund to those students 
economizing on scarce university resources. In such a case the students, administrators and faculty could all be 
tietter off and no one need by worse off. 

The faculty can be separated with respect to their opinions regarding the relative imfK)rtance of the research, 
teaching and filtering functions of the university. Since, for most universities, appointments and promotions are 
based on research output the major apportionment of faculty members' resources will t» biased against teaching. 
Since many outside interest groups demand a minimum of teaching inputs, there is a lower limit of resource com- 
mitment to teaching which the faculty collectively cannot hope to evade in the long-run. There is no incentive for 
the faculty to maximize value-added to the student body; the atle^d fact that bettor students are easier to teach 
and demand fewer contact hours* are adequate selfish reasons for faculty insisting that only the best students 
are admitted despite the fact that maximizing value-added might dictate that relatively few contact hours be 
devoted to the highly motivated "bright" students and relatively many hours to the poorly motivated "dull" 
students. Admission to the university of minority students, on paper underqualified, is often opposed by faculty 



^ fn the United KJnqdom, far exumplo, farulty membors of polytechnics ^ho on avefflqe hflv« tower catibre students than the 
unfverfiitf^^ fnafr»ta»n that Stiidunts demdnns .<nd protHomS aro i4#LH thdt nu dftie I'j «<va»lat>ftJ for rt»s«4ifch. similar arg^imi^nts are ht»*«fd 
from iunt >r coffeg&s^ faculty who have ro^i^rch ^^irattons. 



membm not because they do not teiieve in egatitaridfiisfn but tecause of |:K)tentjal enoroachment on research time. 
Since many students, who play a passive role throi^out thehr university lives, could well be ''ecfcjcatfon" throi^ 
non-labor intensive pedagogies - video-taped lectures, packaged materials, i^rogrammed instruction, computer 
assisted instruction pkis some faculty or teadiing assistant inputs, the questi^ms aris^ why sudi innovations, 
which have r^oven to te, in many instance, equally effectiw as liw instruction, are not enthusiasticaity endor^ 
by the faculty. The principal reason is that, under the present uni^rsity modus t^andi, resources saved by adopting 
technological advances cannot be readily a^^propriated by the innovative faculty memtwr. To take an extreme ex- 
ample, the professor who video taped all of last year's lectures and v\^c^e current students tearn Just as much by 
using these video tapes and are just as happy with tfieir course, (addition non-human, low marginal cost inputs 
may be added) without any actual student-faculty contact, is unlikely to receive full teaching salary from the dean 
or department head. Note that this is not because the output was not forthcoming, e-g,, students learned less, 
rather it is because the standard inputs, especially H^e teaching time, were not employed. The lack of incentives for 
tftnovation exists for the departntent as wdl as the individual instructor. If the dejMrtment can maintain output but 
reduce costs, the departmental bud^t is H^le to be rediH^ and the eff icientry gsAn <^nnot te captured by the 
department collectively. Thus, there is little incentive f cht the department to be efficient or to adopt new i^lago^es 
which reduce costs while nuiintaining output. 

Because of substantial economics of scale it is po^ifale that the full fK^tential of i^rtain |;^agosi^ will t>e 
realized only in either the largest of universities or in the combined effects of several institutions. With minor ex- 
ceptions there has been essentially no effort on the part of institutions of hi^r learning to attempt to allocate 
resources more eff ici^tly cmi an industry wide basis. Again the main reason is that the gains frwi such effoas are 
unlikely to be enjoyed by one of the principal interest groups in hi^er education, i-e., the faculty. The history of 
craft unionism yields many examples where technological innovations are entbraced only when sufficient incen- 
tives exist for the principal actors to acc^t chdnij^ or when the alternative of maintaining the status quo forbodes 
a greater loss in well being* Without a system for efficiently reconciling comfit ing interests of different groups 
with a stake in the university, the actions of administrators in seeking local and national stature, tranquility on 
i^mpus, "efficient and equitable" budgetary allocations, academic freedom, relatively magnificent offices, air 
conditioning and efficient secretaries for presidents and deam, and imf^essive buildings pleasantly landscaf^, 
will often be vociferously opposed by students and faculty. Similarly the interests of the librarian in maximizing 
the number of titles, preserving rare books from anxious little hsmds and housing the collection in an expensive 
building are not necessarily in accord with student and fruity interests. Finally, other employees of the university, 
such as maintenance staff, desiring higher wages and union represmtation, pre-sdiool and child-care centers, 
admission privite^s for their children to the university so forth are competing for resources sought by other 
interest groups. 

Does the existent^ of com^ting interests and the lack of an efficient allocation mechanism dictate that we 
shall continue to bungle through? The answer need not be ne^tive; for example, it is hi^ly likely that the recent 
cutback in university f inan ing has forced many institutions to reduce costs, without any appreciable reduction in 
output. But such savings are obviously limited without the adoption of technological chan^, i.e., the cutback may 
have forced many universities to move closer to the production frontier without shifting that frontier outward. 
Sn order to shift the frontier outward under pre^nt institutional arrangements it will be necessary to motivate 
faculty to beojme innovators through research rewards. Once more efficient teaching pedagogies have been vali- 
dated as cost-effective, university administrations under pressure to cut costs can force these new practices on the 
faculty. The most fruitful research is likely to emerge only if agencies such as the National Science Foundation, the 
National Institute of Education and private, foundations make a substantial committment to the type of lar^ 
scale research which will provide information on the costs and effectiveness of pedagogies with different character- 
istics for students with different characteristics. 

The Shortcoming of Past Research 

Much of the research supported m the past has not been concerned with providing academic decision 
makers with the information required fur them to increase educational productivity. The economist's approach 
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to hi^hm eduGiticm has tmen arBety cchtI ined to the fofiowmg c^eslion: tlWiat is tfie i^op&t aifocation of resources 
between h1^r education and atternative us^? This Gftj^itm has been ^^proached from Mnmrai points of view, 
each of which provides at best an arr^iguous ansvver and none of whic^ begins to address the probtems relating 
to the mtemal efficiency of the hi^r education system. 

The most widety adopted researdi ^;^o^ has t^n to estimate from data on differences in individuals' 
incomes and educational attainments the rate of return to inv^tment in addttKMiai years of higher education. 
Whether sodety would make better use of its r^urces by allocating n^re or l«5 of them to hitter ^ucation 
exceeded or feil short of th^^ on alternative investment opportunities. 

A seomd apf^oach has been to measure the a^egate cumulated eduortiond inv^tm^ embodied in ^e 
labor force for different years and to infer from time series data on a^^pegate output iabw, frtiysicat capital and 
the stock of educational capital the contritnjtion that a marginal addition to the stodc of ^iurattonal capital 
would make to the production of national output. A^in, given ttie costs; of adding to mJucaticH^i capital, it is 
p<^ibte to calculate a rate of return to educational investment to be compared to the rates of return on atternative 
forms of investment with ofovtoi^ implications for resour« allTOation. 

A third approach has been to calculate average educational attainments of employee in different skBI and 
omjpaticKiai categories and, on the basis of forecasts in manpower requirements^ to determine what resources 
must be devoted to higher education if the re<Hii<^^ numt^rs of emfrtoyees with different c^tif ications are to be 
forthcoming. Clearly, if tf^ resources available to higher education fall shcKt of the i^eds caloilatc^ in this way« 
then according to the model it would pay to eliminate the gap. 

Each of these thrro lines of inquiry has been mark^ by serious limitations resulting from inactequacies in 
the available data and untested restrictive assumptions in dw thei^. TTiis paper will not attempt to summarize 
such aiiortcomings since they have been well documented in other publications* However, even if the research 
objectives ODuld be fully attained, they would provide only a limited basis for educational policy, strrce they fait 
to suggest how resources allocated to higher education can be most effectively used. The full complexity of wrfrat 
is involved in the rational planning of hi^r education is only apparent when the prc^lems in allocating 
resources between hi^er education and other eramimic activities are a>nsidered ^intiy with thc^e relating to 
internal efficiency. It seems to an unwise researdi strategy, past regard) efforts notwithstanding, to consider 
the first set of problems and ignoro the second, wlwn the elimination of sut^tantial internal misaliocations vrauld 
have profoi^d influence on the optimum share of resources to be devoted to hi^er education* 

The inadequancy of the results of past research was made painfully evident by the crises ttiat rocked higher 
education during the i^st ctecade. Hit by the dissatisfaction of taxpayers and tfie anti^>nism of 5tu<tents, together 
with rising costs and dwindling resources, educational decision-makers found themselves having to carry on 
with business as usual in the ateence of a moctel of higher education which wuld form a b^is for rational planning. 
It is in this context, that academics, dissatisfied with their own a<xompli^ments and nrore acutely aware of the 
problems of colleges and universities, have begun to take a new look at the (gnomics of education. 

One theme that pervades much of the new effort in the field is that ^^nomics research on higher education 
must find a way to deal with the fact that the output produced by the contemporary university includes numerous 
and diverse components, whose quantities and prices are diffiojlt to measure. The f^^nt lack of meaningful 
quantitative informatics^ creates serious difficulties for educational ctecision-makers, both within and outside 
universities. Unless techniques for measuring these variables can be found or methcKfc of rational decision-making 
that do not depend on this information can be divtsed, the fwrospects for significant increases in efficiency will 
remain poor. 

The fact that university output is multidimensional is not in itself the source of the analytical difficulty. If 
the components of university output wert? like other goods, it would be a relatively strai^tfofward matter to 
apply the calculus of optimizing behavior to the allocation problem and find a reasonably efficient solution. As 



every elementary economic student knows, in orcter for an educational pJanner to determine the optimal wiount 
of the different components of university outfHit, it would be nece^ry for him to have information about both 
technology and preferences. Technology can be dewrited in terms of production relations whidi indicate the 
amounts of each grod that can be obtained from different combinations of inputs. Gh^n the of acquiring 
various inputs, technology determif^s the cost of t^oducing an extra unit of each good. Preferences, on tf^ 
other hand, indicate the terms on which ccmsumers would toe willing to exchange one good for anotha^. Giwn 
total income, preferences determine the price cmsumers would be willing to pay for an extra unit of eadi ^od. 
Clearly, as long as consumers value an extra unit of any component of uni\mr^ty output more highly than it costs 
to be produced, then the university should exf^md production of that components 

In ord^ to apply this simple model, it is necessary to irf>tain ctetailed infwmation on the production rela- 
tions which constrain the choices of university c cision-makers. This will require large scale research effort which 
produce data on the costs and effectiveness of different f^p^ical techniques for different typ« of student. 
Once this information is available it will be po^e for a university administratic^ to find the least^st way of 
producing a given bundle of univmity (Hitputs. What is being sugc^sted is that the inno\®tfOrw which will bring 
about substantial increases in educational productivity will take place only throu^ the con^ination of reliable 
cost-effectiveness research and the application of its results throu^ administrative fiat.* 

Su^ested Re^di Appro^ 

To assess reliably the efficacy of different f^agogies in any subject for stud^ts of varying chaiacteristics 
requires a large scale experiment financed with public funds to evaluate ail promising pedagogies in many ^d 
diverse college and universities. A massive effort is required, not only because of the iar^ number of variables 
involved, but also because there is insufficient variaticm in existing methods of instruction in hi^er education 
to permit accurate evaluating of alternatives. Public fimding is essential, apart from reasons already discussed, 
because institutions may be unwilling to allocate resources to explore the effectiveness of new technologies 
when the private costs to any one institution outw^i^ its exf^cted private gains. 

The number of universities to actively participate in the proposed exf^iment must be sufficiently large to 

i) provide a broad spectrum of student, faculty and other institutional input characteristics 

ii ) allocate the different pedagogies to be evaluated over the ran^s of diaracteristic^ above 

ill) assess different media and instructional mixes and 

iv) provide a sufficient data base on input mixes and various outputs for statistical analysis. 

The exact number of universities required therefore will be a function of the range of the input character- 
istics to be treated and the number of different pedagc^ies to be evaluated. Marginal costs incurred by potential 
participating institutions will have to recompensed to reduce search costs, to ensure adequate numbers and to 
enlist the cooperation within each institution of the course organizer. 

In order to obtain economies of scale in disseminating procedural information, cross fertilization of ideas 
and identification with the research team by the participating institutions, an initial conference with all parties 
concerned would be desirable. It is essential that unanimity of opinion on course objectives and procedures 
be achieved at such a conference. The most appropriate time for the conference would be approximately one full 
year tefore the actual experiment took place. 



* Whtle the f&«istance to change in 99tob}t9hed institutions of higher J^arning may tae TOO Strong to permit sufficient change 
to {Hcreaso etficioncy significantly, now institutions wvithoui vosted »ntero«ts may well In? where the technological progress will occur. 
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U would be the responsibility of the principal re^archers and consultants tu ctesign and f^^p^f^ nnt^ur ing 
instruments of oiftpur. Cost data, both averac^ and marginal, would te r^uJred for each pecbgc^ maly^. The 
total exf^imt»nt| the results are to Ik. Jefinitive and accepted, will have to be carried out utKter rl^rous ex|^- 
mental procedures. To the extent that i^-ifming tiy doing may occur with new tedinologies certain iKHtions of the 
experiment may be red|:%ated during a second year in sela[:ted institutions. After the experimental period the 
data would be collected and analyseil and the results disseminated throu^ publications ami conferences to as 
large an audience as possible. 

One of the maitir probtems associated with rtesignii^ and carrying out such a profect concerns the defioing 
and measuring of etiucattondl output. The failure to deal adequately w?th tiiis problem is erne reason v^y most 
research on cost effectiveness has not convinced the skeptical educator to dtange his teachfr^ practices. Typically « 
for the sake of simplicity, t'xper imenters take account of only one or two dimensions of mitput which are 
re^rded by many to l»e among the less important. Educational outputs can be put ir^ at least three categories: 
increases in students' cognitive skills; changes in students affective characteristics; and the enjoyment or con- 
sumption students obtain from the educational experience. Thus, any maior resean*i effort must take into 
account the range of outputs that fat) into all three cate^ries. 

The first category contains nuinerous elements, many of «*ifch Invol^ the teaming of sut^ct matter. One 
of the first steps that would need to be t^^ken by the researcher is to define the body of material to be indud^ in 
the courses under study which must be common to all pedaspgies being evaluated. At the same time, what con- 
stitutes mastery of the sutvject shouki be clearly enunciated. In economics, for examf^, this would mdi^ the 
following: 

(i) a capacity to follow and sustain an economic argument and to make lexical inferences from 
given information; 

(ii) the ability to set out and communicate to others a logical argument in economics; 

(tit) the ability to be aware of assumptions made implk:itly in the use of an economic model to 
assist a process of reasoning and to perceive how a modification of tlie assumptions might 
effect the conclusions; 

(iv) the ability to understand the mutual interrelations and interdef^rnfer^ies of the various elements 
in an economic system and to take account of them in handling economic problems; 

(v) the ability to understand and explain the economic effectt of important economic institutions on 
economic policies; 

(vi) the ability to make appropriate inferences from quantitative data; 

Ivii) the ability To af^ply to an economic problem the models of economic analysis that are most 
appropriate to if. 

Co^itive learning, however, ts not limited to mastery of course subject matter. It includes as well the 
Krquisition of general intellectual skills such as the ability to reason logically, to communicate effectively, 
assimilate information rapidly, etc. While any good measure of student performance on subject mattsr will reflect 
these ^neral skills, it is important that the resedrcher explicitly define how the learning of general skills enter into 
ag^egate educational output. 

Adequate measurement of the various components in the cognitive category may require a battery of tests, 
ranging from standard f)hjprtiv(' pxamifiatiofis to written essays to the working of complex problems. An addition- 
al reason, mvartably neglected by researche/s, tor a variety of measuring instruments is that students with different 
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character istics perform m a signiftcantiy different fashion in different types of tests. For example, in economics, 
recent preliminary research finding have shown that students pdormmg well in obiective tests do not 
necessarily perform weJI on essay exammatfon and case analyses and that certain student characteristics are 
reliable predictors of how well certain cl^sifiQjttons of stucfents will perform on different types of test 
instruments. 

If such research fiiHiings are substant idted it may be impossible tp construct a single valued measure of 
output which would lie universally appealing. Such ^ index would require a consensus on the wei^ting of each 
of the test instruments and this may be &^attainabte. The fact that a research proposal rais^ such an issue is no 
reason for substituting an incomplete nteasure of output; the issue can be, at le^t partially n^solved, either fay 
accepting the weight of experts in the field or allowing potential users of the r^utts to choose tt>eir own weights. 

F jrther information about course output may be (Stained from the students themselves. While the validity 
of student of»nion on courses and teadiers h^ been seriously qut^stioned as a basis for rewards to faculty, their 
considerable potential for researdi purposes hm t^n essentially ignored. If the produi^rs of educational output 
were subiecf to the same forces as are the suppliei^ of most commodities in our economy tfiey could not afford 
to ignore the opinions of consumer^. While faculty and iKiministrators mig^t remain relatively immune to the 
force of student sovereignty, it may v^ll be tf>at students' opinions of what they obtain from courses and faculty 
are more accurate rro^isures of educational output in the co^itive cate^ry than all conventional tests combined. 
Consequently, a carefully define student questionnaire is a Critical etement in a study of this kind. 

In attempting to measure the efficacy of different {i^agogies^ proponents of increased productivity in 
hitler education need to ^ beyond just the measure of output relatirtg to dianges in cognitive skills and assess 
what may be just as important to human capital formaiiiui namely dianges in affective diarm:teristics. For ex- 
ample, students feeling about and attitudes towards intellectual endeavor and the relevance of subject areas will 
affect their career choices. (Anecdotal tales abound about how famous men diose their fields; Max Plank, the 
famous German ^ysicist opted out of economics because he felt it too difficult a subject for him.) The manner 
in which a course is presented, given the subject matter, by influencing students' emotional outlook, will help 
determine time spent on a subject, opinion of courses and f^njlty, chc»ce of undergraduate and graduate study 
and ultimately career choice. Since such factors could have far reaching implicaticms for both individuals and 
society in general, sound reasons appear to exist for attempting to assess changes in affective characteristics in 
the suggested research approach. Thus the student questionnaires should include questions designed to provide 
mformation on chants in affective characteristics. 

The problems associated with defining and measuring inputs and accurately costing out different pedagogies, 
while much less difficult than those that ari^ in connection with educational output, are nevertheless real and 
substantial. How should different pedagogies be characterized? What are the differences between avera^ and 
marginal costs and between short and long run costs? How can differences in the quality, as opposed !o the type, 
of input be taken into account? While answers to these questions will not be attempted here, they are an 
essential element in any good research proposal on the cost-effectiveness of different techniques of educational 
production. 

The above research proposal has implicitly assumed that universities, to some degree, create value-added in 
their teaching function. Earlier in the paper vw advanced the hypothesis that another function of the university 
is filtering students. If one considers the university purely as a filtering institution, much of the proposed 
research is irrelevant. To achieve efficient resource allocation in such a situation requires only an analysis of 
relative costs and effective filtering potential of different courses and testing instruments. Since almost no one 
beUeves that zero value-added occurs in teaching in the university system, the research herein advocated should 
provide the type of information necessary to evaluate technological innovations and increase efficiency in hi^ier 
education. 
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LEARNING, TECHN0L(X;Y, AND THE POTENTIAL INCREASE OF 
PRODUCTIVITY IN HIGHER EDUCATICHM 



Robert T. FtJep 
Associate Commt^ioner for Educational Techrwlogy 
Director, National Center for Educational Technoi<^, 
U.S. Office of Education 



- Congrdtuidtfons to the orgdni/;*rs of the meetingl 

- The group of participants assembled is imleed impre^ive. 

- Educdtionai technology and productivity, they sound like they ^ould go to^tlier like love and marriage, 
horse and carriage and so forth! 

Today I would like to share some thoughts with you in three areas: 

1. Briefly, how do we defint* this new approach: Educational tedtnotogyl 

2. Any endeavors exploring ways for increasing oroductivity in higher education must take into considera- 
tion the changing nature of the learner as well as having a viewpoint wtiich consicters the f^Jlty and 
administrators also as learners. 

3. Quality and quantity of productivity that we seek must be reviewed in the context of the goals of 
higher educatron yesterday, today and tomorrow. 

Education^ Tedinology 

In trying to define educational tedinology, I am reminded of Jt^ice fitter Stewart's opinion in J»:obetlis 
vs. Ohio; i shall not attempt to define po'-no^aphy, but I know it when f see it. 

Almost everyone is looking for a definition of eduotionai technolc^, some say it's the parts of instruction - 
aids and hardwaie, some say it's the process of instruction - determining needs and strategic — sel^tr^ and 
designing materials - using them with students - evaluating the results. Some daim it's the people of ir^uction - 
the students and the teachers. Most of us would prob^ly agree a ctefinition of educational tedinol<^ indudes all 
of those in a synthesis of the parts - the process - and the people, and perhaps rm^st of would agree that 
beyond the jargon and the buzz words, the term "Educational Technology" means simply the application of moctern 
instructional systems and media to the learning pn^ocess. As sui^, it csn a total ^pmach and not just a bandage 
effort. Certainly, the process ts not exclusively centered around the learner, it musf inclu(te faulty, learners, 
administrators, and suprwrt staff. In short, all those involved in making teaching decisions. Cl^riy then, process 
success cannot be predicated entirely on successful machine or text-related programs. Ratf)er, it will depend on great 
part on the number of decision makers who have been effectively introduced into the process of chMsing tesching 
means and teaching ends. 
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The approach to the subjtfct of the synijK>sium could be along three lines, the!» three lines can be clearly 
sepdrated, although they are intertwined for all practical purposes. During the symposium primarily the approach 
that I associate with monism vvas followed. I believe that the other two ap|)roaches are equally important, at\d 
like to draw attention to them here. 

The three approa^es to utilising educational technology that I distinguish are: 

A. Pr^nfStic Approach 

The pragmatic apprMch would to look at what we have in educational technology that has been 
tested and wtiat has fK^oven to be useful for specific situations. This inventory and evaluation proce- 
dure should of course be nationwide. Tlie secornl step of the praynatk: appro^ would be to identify 
siti^tions in v;hich the suore^fui experiments - which have been selected out in the first inventory 
step - can be r^eated vwth a large probability of ec^al suc^ss. The third step would be the imple 
mentation of step two. 

B. The Pluralist Ai^mmh 

One way to think about the use of educational technology is to ask: What can it do to aid in solving 
the present AD H(K problems we are fm:ing m education, or in another form the question is: How 
does educational ^chnoJogy relate to the present needs for the education system- More in carwerto 
one couhi ixinsider only 

- Whether Educational technology can be an alternative to on-campus instruction 

- Of what use is ^ucationai technology in reducing student unrest, dropouts, etc, 
and concentrate on those types of AD HOC needs. 

I call this a pluralist approach for two reasons: 

- The AO HOC needs could be fulfilled by means which are detrimental to other elements of 
the system and may aeate new problems (O^ical Example: the creation of the nation-wide 
frc?eway network was considered an AD H(X: need in the fifties to $o\^ the military and 
civilian transport problems; it seated the present AD HOC problem of pollution by com- 
bustion engines). Consequently, any utilization of educational technology must consider 
second order effects, 

C. The Monist Approach 

While I recognise the importance of the last approach, the other two approaches should be pursued in 
parallel. However, we cannot lose si^t of the f^t that educational technology f^rovided us with tools 
and techniques whereby productivity questions can be examined. The productivity theme was sounded 
very early in the history of educational technology when B. F, Skinner announced his ''Twira as much 
learning in half the time." You recall that he, Holland, and Arnstein were trying to consider a science 
course for non science majors. Students were utilizing self-paced instructional devices to garner the 
profane dimension of a psychology course (terminology, laws, etc.) which would allow them to then 
sit with Skinner, et al., to explore and interact "with great minds" exploring the sacred dimensk>ns 
and theories of psychology. 

The lecture or seminar was not discarded as a mode of teaching, nor is its demise advcxrated by the high 
priests of educational technology, it is one mode of learning and Should be cl^rly linked to specific outcomes. 
If conveying information is the main objective for utilising a lecture, then, it must be on the defensive. Almost 
everything we would like students to know we can place in their hands via paperback. Plays, too, can be read 
faster than when we sit through an evening at the theater, but reading doesn't take the place of the performance. 
Moreover, lectures provide the opportunity for trying out ideas while they are in the process of formation. They 
are thus part of the teachers' laboratory. The listener is not presenteti with a finished treatise, but rather observes a 
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Ifvtnci miful ai wiwk, cpinnui -m tiiSHjht into straltHjies. not to mention the encri^ flow in exdiange, that tdkes 
place. Ttiis demensK>n (if uittnactmn Iwfween ttHK:ficr ami feanter hasn't even begun to be measufed. Let's take a 
nK>mt»nt ami fxamiru^ this ji».4nu.»f. 

Changing Nature of the Learner 

Most of m ijrvvv ufi m auiiH) *i«)f7 vvh^'rc the prifYie rnoile of communication was audio — the radio, via 
lecture, and the i^iulfxt. hi tunnasts. tht.' avtvai^ child tod^iy spends 16,000 hours viewing a T.V. set during the 
same years ht* s|u>nds 1 1.000 iiuut s liMrnuiq in elennmlary and secondary schools. He is a muiti-ima|^ learner 
accustnmed to learfiiTHi fiom hotti .uhUo jcuI vistj»H fViisentations, He has atso come to expect effectively 
sequenced presentutKHis with j tirfi iisi* c»f color, graphics, and commerciai television quality. 

Thf stmlt!nt». on «mf r.jnipiiv^ ttHlay hdve a f|reater exf^ctation for new and modern af^owh^ that are 
tedinoliMiy Ims^hJ. r'ff«Tt^vj,\ ami rulov.Mit. L<iu(.Jtionaf techfHslotjy provides both the proce^ and products to 
meet such fi<»Cih. 

The explosivo m(:r**a<ie in the volume of communication directed to the individual creates the need for 
resolving thi^ conc«?rn. Jhu^, wm mujht expect learners to deal with an increasing nurrdber of rei^ptors — media 
for receiving communicatiof tapt? rt-corders, copying equipment, etc. 

Tin? growth of a tlif fi-rfi ^tiaied communication system rs part of a larger process by which our society has 
been prmiuciruj a diffcrijntiat«'<l culture, 

Richard Moisel. in the sufiinwfr issue of the Pabfic (pinion (hmierfy, provides consi€Serable data to support 
the thesis that mass medu art* shrinktmj m size relative to the total economy, and that the Sf^cialized media 
tjecoming mcr«-jsin(|ly ifnfwrtant,^ 

Educational technoloi|y is in hannony with this chan^, and si^^ts that one start out by thinking in terms 
of promoting ieai ning, rattier than providing instruction. To do this one needs a flexibility of approach that will 
accommodate the individiial differences existent in all of us. I continue to be intri^ed by flw pot^tial of the 
cognitive mapf^ng ^^ctivitv of Joe Hill and Derek Nunney at Oakland Community CoHege. Their idea of Diagnosing 
students for most effective modes of learning (lecture, seminar, audio-'tutorial, fW-ogramn^d test, individual study, 
etc.) bas-^d upon prior successes is that learning holds great potential. In this approach there is a dimension related 
to the schema sugt^stetl by Keith Liimsden where he outlined a method whereby students wouM be "billaJ 
accordingly" for each mode of instruction selected in relation to its cost (i.e., lecture, seminar, audio-tutorial, etc.), 
the mechanisms woultl be very easy. Think for one moment about a recent visit to disneyland whereby a book of 
tickets marked A, B, C, D w*?re purchased. Certain rides were selected based upon cost — Others, t>ecau$e they 
were potentially more t^nioyable. 

Included m this booklet should be tickets that provide for "dropping out" for a semper or two, to partici- 
pate in the world of work, the ti ckets should be designed, first, to make campus instruction more relevant, and 
secondly, to enable thu student to make decisions relating to curricula that will provide some saleable skills upon 
graduation. A life long learnini|, career education approach in higher education is long overdue. 

The quality of education \s nut tidsed on re(|uired courses; it is based on the interM:tion between teacher and 
student, in essence, we are yivnuj much more freedom to both, but we are holding them more accountable for 
performance, SNt* havt? nut reduced r i^jor through flexii)ility; we have ino-eased it, and need to ifwrease it more if 
productivity is to be int'artt!^<|fuL 

^Miiiicl, ffofijvvof Uu h*4f<!. TN*' f)tH Una of rhp Mt!*;* Mtfdia/' 169*170, Public Opinion Quarterly, summer 1973. 
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Althougti appropriate reinforcers need to be implemented in facuhy promotion constderatlons (i.e., develop- 
»ng an audio-visual course is equal to publishing one book, etc.). morr important considerations are necessary, the 
maiority of today's faculty did iiot learn, nor experience multi-«T»age learning. For most of us, we vi«fe linear, 
print learners, and orbited and continue to orbit in what McLuhan so aptly desmbeeas "The Gutenberg Galaxy."* 
Add to this fact that the bo«fy of knowledge in any field is increasing so rapidly that a faculty member may shun 
any new ideas for involvement in the process of the university to keep his head above water in his own field. 

Consequently, educational technology must enter as a reinforcement, which tt _ faculty member can utilize 
and also develop. Hovaever, time must be provided, as for research, to allo«f updating and media development 
activities, in fact. I would sug^t that in today's worW the ^tdiatical is obs<^te and a "Cinq-Annee" for at least 
a half-year should be instituted to enable the faculty men*er to participate as a learner, as he evolves technology- 
based instruction. Residuals and royalties mi<^t accrue from the production and broad-scale use of television 
productions, media materials, etc. for both the univ^sity and the faculty member. 

Paramount among our concerns for the success of such a project is that adequate facuHy time be made 
available for intensive planning, production, and evaluation. A level of suiH^Oft for participating faculty must be 
provided which will, in effect, remove them almost completely from the many claims upon their time from 
university administrators and housekeeping groups. Faculty should also be retained at a level of support through- 
out the entire course development period (for instance, three years) to permit full participation in evaluation and 
resulting course modifications. 



Further study should be given to encour^ing particpation in, and a«»ptance of, new forms of instruction 
by faculty who have not assisted directly with such jweparation. Presumably they will become in'/olved with a 
project of importance to the institution and to higher edutation, and their attitudes as ywH as tfiose of their 
students houkf be evaluated. 



Any discussion of the creative use of educational technok^y to increase productivity in higher education 
should take into consideration that the percentage of 18 years olds graduating from hi^ school in 1981 will be 
over 3 million, up 10% from 1971, and that the anticipated f»rrant of high sdiod graduates going on to collge 
will increase atjout 6%. The current expenditure for institutions of hi^er educaticwi, public and noni>ublic, for 
1971-72 was S29.9 billion. Based upon wjnstant 1971-7? dollars, whidi we knovw will not be true, we can 
anticipate almost a 50% increase by 1981 to $46 billion. Enrollments only up sM^tly - but exfwnditures 
increasi<ig greatly. 3 

The contributions of the faculty, of course, are only one source of productivity growth. Other major sources 
include technological change, increased capital intensity, improved operating methods and work practices, econo- 
mies of scale, advances in knowledge and shifting work to customers (the learners). 

Evidence imlicates that the bulk of the increases in productivity stem from sourcss that have little or nothing 
to do with the direct contributions of the faculty themselves. 

For example, if 100 men moving a mound of dirt with hand shovels are replaced by one worker using a 
power shovel, the output per man-hour will go up tremendously, 

The source of the increase, however, is not in harder work but the one remaining employee, 

Rather, the source of gair. comes from the advance in knowled^p that permitted the power shovel to be 
invented, technological change m the mvention of the power shovel itself, an increase in capital intensity 
embodied in the power shovel, and the better education and training that permitted a worker who was using a 
ha nd shov el t o learn h ow to operate a power shovel. 

^McLuhan. Mafsholl. The Gutsnbsrij Galaxv, Uni«»f»itv of Toronto Prtt. Tof/'io. Centda, 1962. 

3st«f»st)c» of Trenrti in Educallon. 1961 62 lo 1981 82. Nations) Center for Eo^ onat Statittict, U.S. Offics of 
Education, Waihmglon, D.C, January 1973. 
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Only th« last of these tiMjher skHI — mi^t warrant an increase tied to tiis increased output per man-hmir. 

Taking jnothirr e)(ampit% an imiiortiint source of m-:reased productivity in many industries in recent years 
has invofvi'd a sfuft of work fioni d comjwny's employees to its customers. 

Two outstanding illustrations are the trend tow^d self-service in retaH stores tfire switch to customer 
dialing of ton^-distdnce tt'teptiofie calls. 

Evaluation for Excellence 

Institutifins t»f higher tMluciitiori utih/ing eilucdtiondl technology need courage but should not fear evaii^tion. 
For the very essence o/wa/^/jf#o/i is toefKoar^ the evolkitiim of escalating l&mis of exc^tfm^. Learning systems 
should undergo consider iit>le lievelopment testing and revision with potential tar^t audiences before they are used 
fur ldf^*r audiences. 

Careful examination of training to acttial on-the-job requirements should be considerafcrfe. Faculty and 
student representatives should be encouraged to participate in the formulation of tHwIs and objectives and pre* 
and post evalujtfon. 

We should be willing to examine wry carefully the efforts of. for examii^e. brit»in's open university, empire 
state, sun, and other forms of non-traditional instruction in higher education. By insisting on these formative and 
summative evaluations a great deal of information can be obtained without in^tii^ in the ht^ set-up costs that 
have been "sunk" into activities. However, evaluations that are sensitive and innovative will need to be developed. 
Utilization of adaptations of market research will be required and, in some instance, development of nwr instru- 
ments and approacfies to assessment, we must indeed beware of taking the approach in the anecdote about Mr, 
Green and his canary. 

Chirflenges Ahead 

University personnel utilizing eilucational technology need to be courageous and offer advice and counsel as 
to how to best proceed in the days ahead - particularly as tfiey attempt to res?K)nd to the new clients, the new 
demands of the learning society, and the use of new approaches. 

All I can say as a final item of guidance is "hold your handy in a handy position." The early implementation 
of arwither technology, dviation, provided some caveats, ard thanks to an olden 192) pilot's n^uat from united 
airlines^ I would like to share these with you since they bear some analagous relationshif» with educational 
technology. 

1 . Don't take the machine into the air unless you are satisfied it will fly. 

2. Pilots should carry hankies in a handy position to wipe off gobies. 

3. In case the engine fails on take-off, land straight ahead regardless of obstacles. 

4. No machine must taxj faster than man can walk. 

5. If you see another machine near you, get out of its way. 

6. Do not trust altitiale instruments. 



15 



7. Pilots will not y^mr spurs while flying. 

8. If an emergency occurs while flying, land as soon as you can. 

Think of t^ e accompMshments the last 1 S years have provided in educaftonai technolc^ — imagine what 
the next 15 could bring in higher education. 
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NEEDED: A COLLABORATIVE OPEN UNIVERSITY NETWORK 



Joseph i. Lip«on 
Associate Vice Chanceiior for Academic Affairs 
University of Hiinols at Chicago Circle 

IMTROOUCTtON 

There is a need far «ifi education jl system which transcemis the Itmttatiom of the citfnpus model. The 
diKTumentation of this neeil is extensivif. A ^^r^ral point to make fS the foHc^ng: Unprc^:endented numbers of 
people have arid will have some post-secondary education and ^pirations. These people wiit teml to t>e continuing 
consumers of education for the rest of their fives. They wiU have a ht^ier probabiiity of seeking additfonai education 
Further, they wiU be somewhat more knowfedgeatile about what constitutes good educaiticm. As they took for 
FHTograms to advance their interests^ tt is my contention that indivkfu^ with previous h^i^ier education will require 
advanced programs which in the past have tieen among the most expen^ve pr<^fm; i.e.« training in the laboratory 
sciences, professional and pr e-professional programs. It is not yet dear where the invest n^nt c^Htd ^ill come from 
for the new organizations which are needed. Also needed are new methods of m^keting and finandng education 
from the standpoint of the new dass of student. It is the c^tentkvn of the author that mm way to deal with the 
prc^iem (the onh way which I have been able to think of) is throu^ a G(»Ki£nated ooilMxmitive effort tftifiiing 
advanced educatioiTai technology , among the states and with the aid of ttte federal government. The cottaborative 
organization would have the objectives of developing kmrucP^mat materials for systems of educaticHial technulogy, 
proficiency wsts to facilitate credit for knowledge acquired, - ystems of swifent access to reach those y^ho cannot 
attend campus based institutions, arid an effective information system to guide potential students with regard to 
the alternative opportunities. 

Instructional Materials 

The results of SESAME STREET, THE ELECTRIC COMPANY, ami the coorses of the British Open 
University have sug^ted that the way to truly effective mass of educatic^n^ techncrfogy ts through increased 
investment in development of instructional materials. Materials which are "home made" fwe rmther the qualities 
of o^aftsmanship, artistry, n<x the ^oper incof poratic^ of ^hat is known ^xnit effective instrvctionai design to be 
widely used, Thus the attempt to produce courses on a shoestring Has resulted in giving edi^ational technology a 
bad reputation. 

The author's hypothesis is that poor instructional design is acoept^te to the stiKients so long as it is coupled 
to a social network which en^ds the student's efforts in interpersond situatiom vi4iich sim^n his learning efforts; 
i.e., the campus system works in spite of the quality of teachir^ because the complex set of fniman expectations 
keeps the student working even in periods of discouragement, disintaest or contact with poor teaching. In systems 
which involve various decrees of teaching at a distance, these scKial supports are missing and, according to ttw 
hypothesis, this accounts tor ttw high dropout rates of extension programs. 

One of the remarkable achievements of the British Open University is tfteir relativrfy low dropout rate 
(approximately 20% by one method of calculation). VWiile it is tri^ that the British system uses an extensiye 
course tutor network to support the students' efforts, they also g?ve crectit to the hig^i quality arid excclfcnt 
instructional desiqn of the course materials- Thus the second hypothesis is that as the human network is weakened, 
the quality of the instructional materials must increase in order to hold the stuctents' anention arnl to keep them 
actively involved. 
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Esiimate of Magratude and Cf»t 



Let us estimate that for an effective Open University, approximatdy 300 courses will be needed. These need 
not be devefoped instantaneously, but should be developed over a five-year period. This would mean approximately 
60 courses per year should be prepared to an extremely high stanoard of quality. Let us further estimate that each 
course will cost about SI .000.000 to produce. This estimate come* from both the British experience and other 
curriculum development pnajects. Thus an investment of approximately $60.000,OCK) per year is needed. Even 
after the first five-year period is completed, courses will need to be updated and new courses prepared. While 
$«0.Q00.000 .-wr yedr a small fraction of the cost of higher education, at present there is no method of fiTancino 
such efforts. ^ 

If each state were to take a portion of the task, the financing tecomes much more reasonable. For example, 
Illinois with about 1 /ISth of the population of the United States would contribute 1/15th of the courees. Thus 
we would prepare four courses per year at a cost of S4.aS),000 per year. This would not be too great an amount 
tf the effort was recognized and endorsed by a suitable range of national organizations. It is also conceivable that 
the federal government would support either matching ^ants or a large scale demonstration in one or two states 
Some support is obviously behind some of the work which is taking place h^ and in nther statas. It is simply my 
contentton that the scale and comprehensiveness is not yet sufficient to do the job which needs to be done. Wfe arc 
struggling along at sdbcritical mass. 

I\tote: The estinrate of 30) courses arose from the numter of offerings in typical o^ges and the need to 
provide an adequate range of options, ft is not intended to be a precise number, but only tlw basis of rough 
estimates of effort and cost. Another point to be made is that some organization will be needed to monitor and 
manage the course production network. This will be returned to later. In any case, for the system to work, some 
assurance must exist that each course which is approved (by some procedure) will be used by large numbers of 
participating states. 

m 

Proficiency Tests 

Adequate, widely accepted proficiency tests are e»ential {Mrt of the design of an Open University network. 
They can provide for transferability of credit which is earned in a variety of institutions. They can certify learning 
which has taken place independently, outside of the formal system. Proficiency tests can also serve as placement 
tests, and diagnostic tests, for various deyees of individualized irBtruction, There are serious problems in dewloping 
proficiency tests in some areas of the curriculum and for some kinds of students who have learned much outside of 
traditional course structures. Nevertheless, the cost of developing such tests is much, much less, by my information, 
thar. ' ufse development. We should not allow the chain of the network to be weakened by a weak link in the 
testing component. 

Much work is being done in developing reliable and valid proficiency tests. It is my intuitive judgment that 
there is a better match between talent and the task in this area than in any other component required for an Open 
University network. 



System of Student Access. 



A wide variety of ways of enabling students to obtain instruction are being experimented with. These include 
courses by television, computer based education, independent study, modernized correspondenM courses, 
individual student contracts, courses available through YMCAs and library networks - and various combinations of 
these. Different motivational and instructional arrangements, students wno are handicapped each have problems 
which will require though in the design of methods of bringing instruction to them. 

The exact format of this component is probably least obvious at this time. The future of videotape, comfniter 
assisted instruction, cable TV. are not clear; and their development and growth wUI be important considerations 
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in th^ ctesign of an Open Univei sity system. With adequate sumpKHt and planrrfng such a«:^oa€^ ^ open broad* 
cast by TV and radio, broadcasting vvith on-site recording, educationat rawspepers, and ottKMT fmms of media 
could changi! the ilesiijn equations. S do not mean to impiy that tedtnofogioa* deyelopfi^t can make or break the 
system. The educational and psychotoyical dimensions are much more diff tcuit and tess v^l uniterstood. it is 
true, however^ that the state of uncertainity in technology makes it difficult to zero «n on a particular format. 

Further, some decision regarding media and technology should be SO that the Courses which are 
prcKJiKred will have sufficient number of students usir^ them to fustify the investment. V\ftt8te^ system is 
adopted wilt have to provide for #naximum access by those w4io need an open system. Tht&« axf^imentation 
and developmimt should contir^ut?, a format which offers maximum access and ffexifajUtv without e)u:e$s cost 
should tK? agreed upon. Wf hK' the author has some pref ereni^ ami ideas, tfie ctectsion be brod ufKm tfie 

best engtneering and erfucational expertise that can be applied. 

White aware of th^ difficulties, the analysis made by the British and by I^. David Milter ifuficates that the 
cost (per student course) of 3 system of student acc^ can be brou^t ^li within th^ |[m^nt conventional 
instructiofi^l costs. The importance of tiftis analysis is that (except for cteveic^m^ent costs of the particular format) 
the cos* w be treated on a per student basis and need not be sought ^ a capitat exfwiiditure. 

in summary on this point, the system of student acf^ss should be estifelis^ taking into account the current 
state of the art in educational techno! agy. Flexibility which is not so great that it drim of the a>st of instruction 
is a desirable obfective. Standardization of the media format wtttch attows us to reach man of the potential stuctents 
is another desirable factor. There is a traiieoff between flexibility and standardization of format which must be 
worked out. 

Information System 

Students neeo to know wtiat educational ixograms are available to meet their personal <rt>jecttves. They may 
also need to know something about their goals. If they are interested in fH^>arif^ fof a car^r, they need under- 
standable descriptions of what a oiven career entails, what kind of life it offers. Peopte nmi ttwh better informa- 
tion about job prospects in various lines of vw>rk, Oh^ they are certain ttiat tf ley want to serioi^ty explore a »ven 
Career, ti le system should provide the client with information abcnit M^t kinds of preparation can be (4>tain^, how 
long it will take, how much it will cost, what the entrance requirements to the various programs are, what the 
attrition rates are, etc. 

Much work has been done along this lif«e. The most advanced work vrfiich 1 am faniiliar with has been carried 
out in Oregon with a computerize ) career education information system. The system should inform of all 
appropriate programs: public schools, private schools, proprietary schools, on the job training programs, f»^ograms 
of the armed forces and the federal government, learning exchange f^-o^ams and other nontradittonal and open 
pro-ams. 

The most difficult ^yari of keeping such a system is that of keeping the information complete and up to 
date. New programs are constahtiy being initiated while otfwrs are dosing. Vital courses may be oversubscribed 
and credential requirements may be overly subscribed and credential requirements may suddenly be changed in 
certain professional programs. Because of this problem, it seems that a computer based system which feeds a net- 
work of terminals is the most effective model. 

An additional consideration is the role of guidance counselors. It is my judgm^t tfiat soma human guidant ! 
is needed, but I am corrcerned about the past history of profe^ionalizing ^idance to the point that it becomes 
very expensive and the counselors become more and motB remote from the people whom they should serve- It is 
my personal conclusion that the emphasis should always be on educations. The individual should learn about ^e 

sources of information jn<l how tii int^Tprt^t the information. 
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The cost of such an information system can be kept sur^^^istngly low. The Oregon System Of^ales at a cost 
of a faw dollars per user pw year. 

Art obvious possibility is to couple the information systenn with a testing service. The tests could be a wide 
variety ranging from aptitutio and vocational interest tests to proficiency tests which can serve to obtain actual 
course aeifit. The services provided by such a network of information centers could be financed in a variety of 
ways, f^^obably some drran^ment in which fees are adjusted to agoid some trivial use while not inhibiting serious 
clients would be ideal. Some subsidy of the overall operation of the infomiation system and some way to forgiw 
the fee for poor individuals would seem to i>e in the iHJst interests of society* 

Organization 

An administration and coordination office is ne^ed. Tasks, responsibilities and deadlines will need to be 
ayeed u|)on. Schedule adherence will r\eed to be monitored. Tests and instructional materials will n«d to be 
reviewed and tested for effectiveness. Completed products will need to be disseminated. TJhjs there will need 
to be an administrative officer and staff for each of the mafor comfK^nents: instructional materials^ proficiency 
tests, student access system, and information system. 

In order to keep the network responsible to the needs of the hi^er education community and the states, some 
appropriate bcKly with representation from the entire anintry will have to be selected or named do :el policy, 
f^crhaps, an elected Board of Oir^tors of reasonable size couM be practical policy setting ^oup. 

Such an arrangement immediately raises, in my mind, qwstions of autonomy and bureaucracy. I would hope 
that knowledge of organizations and organizational theory to avoid some of the dan^rs. In any case, my logic goes 
something like this: We need an open university system. An effective system needs a large inv^tment of effort 
and capital. No individual institution or even individual state is likely to allocate the resources needed to do the job. 
Therefore, some means must be found to unify the strength of the higher education community. 

The possible difficulties are the following: 

It will be difficult to get a sizeable number of states to agree on a common formula for allocating resources 
and a common format for adr«inistcring these resources. 

The federal government will be reluctant to set up a national Optn University effort of this magnitude 
because they would be charged with the interference in what are basically state operations. Even if they only 
fund and do not attempt to administer, they wili be open to charges of unfairness in the allocation fuimula unless 
the funds are given in some relation to the size of the states. 

It will be impossible to effectively administer s'jch a confederation. The British Open Uni\«rsity, it can be 
argued, works precisely because it is unified and forused. 

In answer to the al>ove difficulties, the following fKimt can be made. The individuals who are attemptmg to 
set up an Open Universities are aware of the needs which have been outiinet^ The individuals I have spoken to have 
been uniformly enthusiastic about the basic idea which has been proposed, v^hile it is not clear that enthusiasm for 
a far off idea will be translatat^le mto action, it has been enough for me to continue to work on more detailed 
planning at our mstitution and our state. 



SCENARIO 

It may help the concept mot^i concroti' if a brief sketch of the Wdy the system would work is given 
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Mrs. Jane Guf ofski tiNirnetJ wtien she was d \imof in coMeg<^. »>e had avera^ to gcKid ^ades. Her major was 
in history. VWien she mai ned, she iniit school to work to supfKsrt her husbanU who was in graduate school. Now, 
Mrs. GuroJski tids Ihmmi out of school for 10 years. She has young children in school, but sliw v^ois to continue her 
edticiition witfi j view of t«»f n^uiii ;i i ar*"-i with fnort* satisfaction and potential advaiKement than any she could 
manage with her present ievi'l of tri*tr«ng. 

Mrs. Gurulski lUitfCwl dit «i(l m ttn? duily pdper encouraging people to make use of the career education infor- 
mation system. The ad proinistul that fui d char^ of two doll<^ that one could explore c^eer |»>$stbilities and 
find the educational pfoqrams to prepare one for a chosen career. Since one of the information offices was 
)ocatf!d in a neatby hifih sdniul, Mrs. burofski stopped in one evening. (The hiformation center kept hours until 
10 p.m. nach <lay aiwl was upon *>m wet'kcnds to ^Micourage use by people who work.) 

The information off lo* was Ut*m\ manned by a young man who was earning money for his education by 
wiHking at the center. In order to man the center, he had l)een given a three-week orientattc^ and training 
program. A hnjlily traifwd professjfinal v^os iv. diar^ of ten of these offices and the professional had r^fwjnsibilittes 
to both tht? |)t'opli* who mdfin*»ti the offices anci individuals who needed si^dal f<M7ns of a^istarioe. 

The young muti askthi Mrs. Gurolski if he could help her. Somewhat awkwardly, she said tfiat she wanted 
some way of establishuvj d career jini f>ossibly to find out hew she coukl complete hex education. The off ire aide 
asked her to sit down at a computer lerinmal and to type in the word, START. The computer program alternately 
asked some questions, <jdve Mrs, Gurolski the chance to ask some question^ and offered lists of alternatives. 
Explor;)tions of mtercsts, limitations, past prcfiarations, finandal considerations, etc., ^adually narrowed the 
number of desirable job targi?ts ami feasible educational programs suitable for Mrs. Gurotski. 

Mrs. Gurolski became convinceiJ that she should obtain her &scf^or's Ctegree in order to maximize her job 
chances. By taking each term for one year, one course at a local junior college, one C^ien University course w^ich 
was available for home study, and one other Open University course wWch required her to come to a local learn- 
ing center once a week, she could obtam her degree so that she would have several options ujKin completion of her 
undergraduate woik. The courses included criminal sociology, general administrative theory, technical writing, and 
urban political science. 

The forms whi<:h were neiHled we»e all available at the information center. Upon completion of the forms, 
they were imme<1iat4?lv rrviewud by the professionai guidance person who was available to scan them by picture- 
phone, even thouqh, he was m an office about thirty miles away. 

Mrs. Gurolski was assigned a personal tutor from the Open University whom she a)uld call for assistance and 
who would monitor her progress and call her if trouble si^ls ar(»e. In additk>n, she was given the names of people 
who were practicing in the careers that she was most interest^ in and shewed tf>e most aptitude for. 

In each of the courses she signed up for, she was given a printout of five people who were taking the same 
course who hved nearby. Her r>ame was also given to the other five. A letter encouraged the six students to meet 
and form a self-help group. She was aho qi i/en the names of individuals who had completed the course who Iiad 
indicated they would t>e wdhng to holp students in the course, 

Mrs. Gurolski was notified of acceptance into the courses by mail at her home. Thus, through the information 
center, she was abU> to renew her academic program by a Single visit to a single office. She was able to establish a 
program which fitted her schtnlule limitations and her financial Constraints. She elected to obtain her B.A. degree 
throu^ the external degree granting authority of the Open University in hei state. 

In each of the courses she took, she was initially given a placement diagnostic t^t. This test informed her of 
weak elements in her h,ick<|r(Minfl arwl how these could be remedied. They also identified her strong points which 
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she shouW use to make her pfoyam more efficient and useful for her. If she had done exirenneiy well on the 
tests, she coukl have tested out of the course and either taken a lighter load or taken other courses. 

During her year of study, she made appointments with peopltr working in her fields of intemt so that by the 
time she completed her degree, she hail a dear strategy for her next step. 



CONCLUSION 

In the author's judgement, an Open University network is urgently needed as we move twvard the year 2000. 
B«*iind each of ttie points raised there is an extensive literature which can give detail and structure to the over- 
simplified version presented here. If the f^oyam begins, there will be tremendous number of practical prctolems 
which will require planning, analysis, and accommodation. Without mirHmizing the problems, the opportunity is 
such that we have the technology, the knowledge, the talent and resource to produt^ this new alternative in 
higher education. 



COURSE MATERIALS FOR ALTERNATIVE INSTRUCTIONAL SYSTEMS 



SYNOPSIS 

The institutions will plan for a long term cooperation in the a) production of course matm'idls, b) systems of 
studeiii access, c} proficiency teste, and d) evaluation of the system. The intention is to serve large numbers of 
unserved potential students in the ^eater Chicago area through the use of media and educational technology. 
In this proposal we seek support for planning and for the preparation of one course of high quality. The institutions 
will seek additional support from other agencies and will build a major cooperative effort for alternative 
instructional systems. 

Justification of Profmal 

There is a large number of potential students in the greater Chicago area who need and desire educational 
experience but who cannot be<x>me part of the campus system. On the other hand, there is outstanding talent and 
experience in alternative educational systems in the cooperating institutions. For example, the Chicago Television 
College has been a pioneer in televised instruction. What is needed is a way to coordinate the extensive talent which 
is present and to involve a sufficiently large number of students to justify the required investment in instructional 
materials of high quality. This proposal is a vital st^ in the development of an open university system for the 
Chicago area, and hopefully, the State of Illinois. 

Televised instruction, extension courses, open universities, and other nontraditional forms of higher educa- 
tion have been struggling to become a major, respef ted component of the educational system. One difficulty 
has been that the instructional materials have not teen of sufficiently high quality to attract use by institutions 
outside the one that originally produced them. As a result, the number of students involved has not been sufficient 
to juftify th^» investment costs which would insure high quality. Sesame Street and the British Open University 
have demonstrated that impK>rtant progress can be made if the circle can broken with improvements in quality 
ani through innovations in the way that students can ^in access to educational experience. 

In the decentralized world of American higher education, the innovation required is a system of collabora- 
tion and coordination in order to avoid duplication which forces mediocrity on eacli duplicated effort and which 
limits the si/e of the audienci? for each course. An important step can be taken by cooperation in the Chicago 
Metropolitan area among the public instituti 3ns of hi^er education. Among us^^we have the (X)mbined resources to 
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develop outstanding course teams of subject expet ts« instruction*^ de^Jigners, and educatioi^i system techrK>lo- 
^ests. If we wiirk together so that each institution uses the course develop^, we wHf ctealing with numbers 
which will f esult in an esccefleni cost/Uonef tt ratio. 

Eventually, by starting at the local levd in an area of high population density, we hope to develop a rmdel 
of coUaboratiofi which will serve for expansion to the state and national levels. Several schcK>ls armind the country 
have expressed interest in this concept. 

Pf oleet or Program f^rector 

The program will be guided by a representative from each of the cooperating institutioi^. The representa- 
tives presently are: 



Dr. Ted f. Amfrews, Dean 
College of Environmental and 

Applied Sciences 
Governors State University 
Park Forest South, Illinois 60466 
(31 ii) 563-2211, Ext. 300 

Dr. Anthony LaDuca 
Project Director 

Office of the Dean of Education 
Chicago State University 
Ninety -Hi fth Street at King Drive 
Chicago, ilHnois 60628 
(312) 996-4433 



Dr. Joseph I. Lipson 
Associate Vice Chancellor 
for Academic Affairs 
University of Illinois at Chicago Circle 
P,0. Box 4348 
Chicago, Illinois 6(%^ 
(312)99fr4433 



Dr. James Zigerell, Dean, TV Colle^ 

CNca^ City Colleges 

3525 W. Petm^n 

Suite 17. Terrace Level 

Chicago, lilmois ^>659 

(312)S8&2(M>0 

Of. Frank Vogel 

Assodate Professor of Elementary 

Education 
Northeastern Illinois University 
Bryn Mawr and St. Louis Avenue 
Chicago, Illinois ^625 
(312)5834050 



Under this proposal, a project coordinator will be hired. The proposal was prefyared by Dr. Joseph Lipson. 

Period of the Program 

January 1, 1974 - December 31. 1974 
Objectives of the Program 

1 , To develop a plan for coop&^ative ^tion in: 

(a) the f.'evelopment of course materials for systems of instructional technology, 

(b) the development of systems of student access to accommodate people who cannot easily avail them- 
selves of the campMJs-based system. 
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(c) the development of proftciency tests whidi will permit students to demonstrate competence without 
tdking the course and which wilt serve other purposes in the instructiundl system, and 

UO the development of a system of continuous and overall evaluation of the cooperative effort. 

7. To develop a sif^ €Ourse a$ a practical exercise of the system, TNs course will be product by a course 
team with members from each institution. The decision regarding the course to be produced will be maife by the 
policy council of representatives from each institution. The course development project will be administered by the 
project coordif^cor who is to be hireil 

3' To develop a model of cottaboration for an ahernative cducantw systern. 

4. To prepare proposals to othm- agencies to enlar^ the cooperative effort. With our central and pivotal 
location in a major state with a mixture of a large urban nucleus and rural areas, we a>uld dewlop a major industry 
of course development in the state. 

5. To have an impact upon the effectitmmss and efficiency of campus instwction throof^ the devdt^ments of 
theorojecl Many of our students work, and this reduces the ftexibility of their scheduling. For other students, the 
traditional mode of instruction, we hope to be able to blur the boundary tetween the campus system and the open 
university typw of system. By offering a wide range of alternatives to ail students, each student could find an educa- 
tional program most suitetl to him. 

Detailed Description of Program 

1. Developnwnt of Pfan for CoHatoration. 

(a) Course Dev^opment. Typically, educational technology has failed because the course materials did 
not match the hardware either in needed quality or supply of useful courses. It does little good to have 
sophisticated and exfwnsive hardware systems (dial aaress, closed circuit TV, etc.) if there are no good 
materials to use with the system. Thus an important part of the project will invol^ planning for 
extensive, excellent course development. ^ 

We shall identify the subject experts who have the capability and the will to become deeply involved 
in course development for open university systen^. We shall identify the courses whxh can most benefit 
from interaction with concepts of instructional design and the application of instructional technology. 
For example, all of our institutions have the need for individualized f^ograms in basic learning skills: 
reading, writing, mathematics, and logical analysis. In other cases, certain large enrdllment courses are 
vital to a number of career patterns. Many of these large enrollment courses could be made more effec- 
tive through the systems we have in mind. 

We shall explore and feed into the planning the probaMe acceptance of courses by students (both on 
and off campus) and faculties. We shall also explore appropriate metha<1s for dissemination to other 
institutions of the courses developed, 

(b) Systems of Student Access, We have among our institutions many of the pieces of extensive outreach 
systems of student access: open broadcast television, closetJ circuit television, video-tapes in libraries 
and YMCAs, community and community college learning centers, extension systems, courses by 
newspaper, PLATO computer based education, indepeoilent study, CLEPT credit and othf?r proficiency 
and placement testing systems. None of tiiese efforts have reached cntical mass either in nufnber of 
students who can be accommodated, in the range of courses available, nr in adequate cost control. 

Wi? must plan fur cooperi?trve use of certam facilities m order to make? them feasible for present 
development. For example, d we can set up learning centers with computer learning terminals, audio 
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afid video tape units, counselors, and tutors which students froin any cooperatirtg institution can use, 
we may yet ft»vels of usage which would make this fnod«l extremdy attr^tve. If we can improve thf? 
quality of open hroisdc:asC instructiofia! matenals m certain basic courses, we may attract greater 
course enrollments. 

There is a large nunil^r of specific issues to be worked out in the plannif^. Standardization and 
trends in the hardware must fc^ estimated. Acceptable initial and maintenance costs must be estabNshed 
and the organization of counselors and tutors must bf> carefuiiy worked out. Overall unit costs of 
instruction for different models of access munt be develof:^. Specific sit« must be found aiKl co- 
operative agreements for use of facilities must be prepared and agreed Ufwi. 

Schedules of development, funding requirements, and impicmentation will be prepared. The individusris 
responsible for different elements and new staff needed in different nrnxiets will listed. 

(c) PfOficmrK:y Tests. Proficiency tests are important to the concept of alternative education. It is 
important to the pfutosophy of the open university that a student who has learned scnnething should 

able to receive credit without having to ^ throu^ unneeded instruction. This re<^ires proficiency 
tests which meet standards of the faculties of participating institutions. 

Well designed proficiency tests can serve other important purposes. Tl^ can be used as i^acement 
tests, finai examination tests, and diagnostic tests for individualized ir^tructton. In general, they €:an 
make the entire educational process more ef f ident and less frustrating to tfm student. 

(d) The plan will include a continuous evaluation plan. The primary evaluation for cour^ materials wilt 
lie in their ability to have the student reach desired levels of competemre, and acc^tance by faculties 
on the basis of the academic soundness of what is being taught. These will assessed by test sabres, 
enrollment and retention rates, and faculty judgements of the rr^terials. 

For other aspects of the plan, evaluation o-iteria and methods will be prepared* in general, the best 
evaluation of the project will \^ the extent to which further a>of:^ation is ^nerated and realized. 
To this end, the evaluation component of the final report will dooiment the extent and prospects of 
long-term cooperation among the institution. 

2. Devefopment of a Single Course, Planning can become disconnected from reality. For this reason, we 
f^opose that a single course be developed in order to provide an important task in which various pr<H>lems of co- 
operation will have to be worked out. By the end of tne project year, the first d«ift of the course will be completed. 
We would like permission for the revision and final version preparation to extend t^yond January 1, 1975. No 
additional fun6s will be required, but the expenditure of the funds should be stretched for this component of the 
work. 

The course materials will be designed to be adaptable to as many different media formats as possible. 
Motivational aspects will be part of the course desij^. Selection of the course to be developed will be the first 
order of business for the policy council if the proposal is approved and funded. The course i^oduction will be 
administered by the project coordinator. Key academic decisions will be made by th0 course team which will have 
a course team leader nan^ed by the policy council. 

3. Dewlopment of a Collaborative Model, There are many details to be worked out if large sr^le cooperation 
is to become a reality. Acatiemic faculties and administrations of each institution have certain f^ocedures which 
require prior approval for new courses and new ways of dealing with students. In ad<)>tion, certain legal and 
monetary questions will arise and need to be worked out. For example, if government contracts are obtained, over- 
head will need to be hharad according to some acceptable formula. If the program is u^d commercially, authors, 
and the participating schools will have to figure out a formula for royalties. Still another important function of the 
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model buitdmg will be the desion of ways to stimulate and reward collaborative activity by the facuttt^ of 
the participating institutions. 

4. Preparation ofPn^asafs to Other Agencies, individuals m several governnsnt agencies and foundat ions 
have expressed interest in proposals for research and development in the area of educdtional t^hnology which 
involve cooperative efforts among several schools. Chances for funding are usually improved if prior work can be 
documented in a proposal. Therefore, the evidence of cooperation among th- ir^titutions and the support by the 
State Board will significantly imf^ove the chances of being funded. Several national par«ts have called for 
regional course production and educational technology centers, Tliis effort could grow into such a <^ter for the 
upper midwest area. 

5. fnu^t upon Campus Instruction. There are many ways that on campus instruction could beni>f it from 
the products of this effort. The course team will be learning new a^cts of iretructior^l dest^, learning psychoio* 
gy, and ways to use the media. Thus it will have a faculty development aspect. Because being named to the course 
team will have a high status, course development and teaching wii! be looked upon with jester favor by otfier 
faculty members. 

The techniques of course development and the use of media have important applications to courses on the 
campus. The visibility of the cooperative effort should increase the cham^ that th^ techniques will be u^. 
The course which is developed will probably be used on the campuses in order to i:^ovide greater flexibility and to 
deal with campus instructional problems. 

Under current or sli^tly modifted administrative procedures, students will tie able to ^leci for themselves 
a program of study involving both external courses and campus courses. This aspect of the osoperative progfram 
would increase the options and the opportuniti^ for many stuttents who work or have other r^fKmsibiiities 
which keep them from becoming full-time campus students, 

6. Contributi<H}S by the Institutions. Since the interinstitutional proposal does not provide any overhead, the 
cooperatmg institutions will be making this obvious contribution. 

Facilities. Media production facilities will be provided by the institutions inwived. These include extensive 
studio facilities at UlCC and the City Coileges of Chicago, as well as those of other institutions. We have not asked 
for secretarial help and will provide this from internal resources. Travel costs will be assume! by the institutions. 
Supplies and technical costs will be considerably greater than the amount requested. While released time to the 
extent of S50,(X)0 is requested under the grant for the course team, additional support will be forthcoming from 
a v;*rietv of experts on alt the cooperating institutions. Individuals such a Dr. Zigerell, Dr. Andrews, Dr. Markfe, 
and others will contribute their knowled^ and experience to the proiect. The project will, in fact, act as the 
catalyst for focusing talent and resources from all the institutions. Only by such cooperation can we begin to 
realize the promise of educational technology. 

?. Evaluation Procechires, Ev?tustion records will be developed by the project coordinator in the following 
categories: 

la) Milestones of agreements for collaborative development of alternative education systems. (Cross registra- 
tion agreements, agreements for course development, cooperative |:^oposais, agreements to share 
facilities, etc.); 

Proposals submitted to other agencies. Proposals approved; 

(c) Milestones of course development; 

(d) Review of the quality of course elements by experts. Results of formative evaluation of course elements 
tried out with students; 
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M Evufence uf mnwct upon cdmptis comfnunitit*$: use of new techniques, increased interest and activity 
in course cicvciopment ami new apY^oadies to teachuiq, reailocatton of resources into course develop- 
ment diHf teactiiny^ chanqe in attitude. Evidence will indude articles, student newspaper notice, 
letters of iiuju*^ Jttt^ndunCiJ dt nieennqs, afH)l>cdt(uns for participation, etc.; 

(f ) Documents ilevoiopiHl and afiiHoved by the policy coui>cti. These will include, in particular, the 
planning tlociimcnts and the iTiodel for cooperative action. 

Budget 

Project Coordinator , $20,000 

{Salary for one year) 

Faculty and tit::ff, releasetJ time to pre^wrc course materials SO,(KM) 

(5 X $T0,O(M3 for year released time for five individuals) 

Sisppfies and technical ^rvkes for the production of one course lO.CKK) 

production 

Total 580,000 

Exf^nation of Budget 

Project Coordinator. By havinti fumls for the project coordinator, the policy council of the participating 
institutions will be able to select an individual whose primary allegiance is to the project and not to any single 
institution. Thus, in turn, will dominate or preferentially benefit from the collaboration. The individual will know 
that his career will d^etnJ upon the growth of cotHJ^ation among the imtitutions. 

Course Team Support. These furwis will allow us to attract one outstanding mdividual from each of tl>e 
participatmg institutions. The course team approach has worked extremely well in the British Open University de- 
velopment, and these funds will allow us to explore a model for this kind in the cooperative mode. 

Supplies and Technical Sen/ices. This relatively small amount for supplies and technical services will give the 
project some flexibility in the purchase of items which are not available from one of the participating institutions. 
<f we need some animation, special art work, the time of outside reviewers, more film or video-tape than we can 
obtain from th:» stores of the institutions, these funds will allow us to act quickly in the interests of the project. 
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GROUPED AND B0Ur4DED 
LEARNING ENVIRONMENTS 
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READiNG, WRiTING, ARITHMETIC AND COMPUTING 



Arthur, Luehnnan 
Professor of Physics 
Dartmouth CoHege 



The Pfoper Rcrfe of tfie Com|»iti?r in Education 

In ihe v/dst mrtjoriiv uf uiuvtjrsiiies the computer is virtually isolated from the prUidfiai clientele, the students. 
The computer may mm vtj thu wtHls of aiiministrative data processing, such as handling siuder«t reoKds or the 
biweekly payroll. It tnay a I jtiorijtory instrument for the computer science department. In tfte most fK^tigious 
universities the computer scrv«?s exctusi\reiy the research needs of a small p'oup of "big u^rs"' with srant money. 

Educational comfHitinci is qiven short shrtft, and when it is served it is often done so in a misdirecter! 
fashion, based on the following f<jlse premise: 

Educational com^HJtiny re<|uires only a fairly simple computer cap^ie of running compute assisted 
instruction {CAD drills aiui other teadiing programs. 

To many people m decision making positions, educational use of the computer is equivalent to CAI - nothing 
more and nothing less. University ailmmtstrators look to CAI as a way of reducing the cost or increasing the 
efficiency of the teaching staff. Teachers look to CAI to relieve them of tedtus jobs, stich as teaching remedial 
English or French verbs. Even students ohen show a preference for CAi over oonwntional instruction, and some 
studies show that they learn better from CAI. 

Nevertheless, the premise is faulty for two reasons. First, dmnjmented examples of effective CAi are few. 
institutions have based a decision to install a computer to deliver CAi on the belief that there now exist thousands 
of hours' worth of good CAI programs waiting to tie run on their system. Acnualiy very little exists and much of 
It is so system dependent thai it js nf>t easily exportable. 

Even if It were true, however, that do?ens of hi^ quality, well tested CAI courses now existed, there remains 
a second reason why the CAI prrmise is a faulty one. It presumes that the only educational role appropriate to the 
computer is that of the teacher, the deliverer of instruction. It is because of this presumption that one so often 
hears the educational use of computers described as being competitive with conventional teaching. Such discussions 
usually fall into an uncertain (Quagmire where questions of cost -effectiveness and laborintensiveness are debated 
endlessly. 

To become boggetJ ciown m such an argument is to miss the whole point of educational computing. Instead 
of focussing on possible ways m which the computer might affect the content SfHf subject rmtter of education, 
one is limited to considermij only the economics and technolc^ of defiv&'ing instruction of an otherwise perfectly 
conventional content. I Ho not sjy that the economics and technology of delivery systems is unimportant to 
education; rather that there is more to computing than delivering instruction. 

it IS the principal thesis of this paper that computing has new and uniqi^ implications for teaching and 
learning. CAI only begins to reveal the smallest part of those implications arKl should not deviate our attention 
from grander potentialities. Worse yet, a computer system optimized to deliver CAI drills may be very poor at 
handling the more inij.xirtdnl tHhHMumal functions to t>€ described later. The ironic result may be that the 
<x>mputer system acquired for education^jl use is inadequate for educational use. 
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Upon what premise, then, should uruvmity computing t)e founded? If the campus computer is oot to be 
merely an administrative data processor, if it is not to be just an adjunct to research, if it is not to be mainly a 
laboratory for computer science, and if it is not to be simply an automated drill instructor, then just what is it to 
be? The answer to these questions derives, irrmy opinion, from the following promise: 

Computing ts a new and hindamental intellectual resource in the same sense that reading and 
writing are fundamentai intellectual resource. Universities need computers for the same 
reasons that they need libraries. 

Tills may seem a mildly bizarre claim. After alt, a computer is an artifice, a mere machine, a piece of technical 
apparatus. Isn't it pretentious to identify a box full of wires and transistors with the world of intellect? I think not, 
AH of the objects of intellect are artificial. What could be more arbitrary or artificial than an alphabet or a system 
of numbering? What word has intrinsic meaning? Yet out of these artificial elements people have built all of the 
linguistic and mathematical edifices that we know. 

Computing likewise begins with artificial, intrinsically meaningless elements, of which the machine itself is 
the least important: variable, constant, instruction, branch, conditional branch, input, output. At a slightly higher 
level, there are more artifices: algorithm or procedure, iteration, subroutine, recursion. Just as alphabets, words, 
sentences and paragraphs are the means by which we give linguistic expression to certain of our ideas, in the same 
way the elements of connputing give us a new form for the expression of old and new ideas. Like language an,d 
mathematics, computing is a way of thinking' . 

Examples of non*CAI Computer Use 

A few examples may clarify the t^ilosophy of computing prevalent at Dartmouth and a Rowing number of 
other schools. 



Historically computers found their first applications in the physical sciences, so it is not surprising to find the 
first interesting educational applications there as well. Eight years ago this fall a group of f^ysicists convened at 
Irvme, California, to discuss ways in which computers might affect f^ysics teachings. Several of us came away con- 
vinced that in a few years the entire pedagogy of freshman mechanics would be transformed. 

In the past a dear understanding of Newton's Laws of motion had required a firm grKp on the fundamental 
ideas of the calculus. This is true because Newton's SecorxJ Law is usually represented a& a differential equation. 
Now, the plain fact is that fpw freshmen have enough experience with calculus to be able to tolerate a presentation 
of motion in terms of differential equations. As a result all textbooks and n^st teachers have settled for using the 
symbols and nomenclature of the calculus; but very rarely have they actually asked students to solve for motion 
by using the calculus, {Check the problems in any textbook, for example.) 

But the fact of the matter is that Newton's Second Law has an alternate representation that avoids the 
whole calculus problem. It can be thought of as an algorithm or procedure for determining^ the position and 
velocity of 3 body a "small" time later, given the current position, velocity and the force acting on it3. T»ie £teps 
or the algorithm are simple algebraic statements that a tenth-grader can undei^tand. Of course, an algorithmic 
approach only makes practical sense if one has a computer handy to carry out all the tedious calculations. 

The payoff of this fundamental change m j^agogy is enormous. For the first time it is practical to teach the 
essential idea of n>echanics instead of cheap imitations^all those awful inclined planes and pulleys. The subject 
matter, motion, ran be taught s^neralty, without having to worry whether a particular forc^ law will lead to a 

'a, W. Luahf mann. "Shoukl the Computer Taach the Student, or V<c« Vofw?" Proc. AFlPSSJCC 1972 Vol. 40 pp. 407 410. 

-^Th^ Computer m Physits. Report of the Conference on the Vim of the Computef in Undorgrariuato Phyiict Instruction, 
Univerffty of California. Irvine. 1965. (Cornm»«s<on on Coltege Physics, 

^A, M, Bofk, A. W. Luohfrnann, J, R. Rohson, Introiluctory Computer Eosed Mechanics ', (Am. Insl, of Phyiic* Infofma- 
iton Poo*. SUN Y Stony Brook. 1968), 
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malhematicaiiy soluble piubtem. The aiymithn^ applies to an/ force law. ami if even gen^raiiTes e^tly to haiHjtt^ 
two and three UifYieiisiondi snuations. Our freshmen esf^^ly tike to program the C(»n|iuter to display anifiDat 
satellite orbits and to find the nqht launching condiltons to send a vehtde to tN? moon and badk -somelhing 
unheard of under the tild |HHlac|o«|v 

Students m the social sciences at C^rtmouth vie wUH those tn if^ sciences for Ingest oHriputer t^te. In 
sociology a stud«3ni, using the IMPRESS system^, le^ns to do at^ntitattve research on actual data, not different in 
kind from th^x carried out by professionals, ^tttng at a terminal tt«e sturiem can ie&dh any of siHty different data 
bases ami can direct a :iequence of inquiries aimed at finding out, for enamfie^ who in ttie siting of 1973 believed 
in capital punishment. He c«in look at overall results afHf he can tie^n to crc^ t^Hiidte belief in coital punishment 
with age, sex, income, race, religion, etc. Inside a few minutes su<^ a student researcher mil have formulated 
several Hypotheses and checked them out agdmst tfie data. 

As m the phytnics examiUe, we see here a stu<fent doing sociology ar^J m>t merely tearing ^boul tt or foetng 
drillufl on it via a CAI los$on. The situation m music is similar* 

A joint project tietween the music department and the engineering school has been itie development of a 
musical terminal - a computer-ctnven electronic music synthesiser, no^ in use vvith over a hundred students^ . 
Sitting at the terminal a student can specify a compositic^ by typing a sequence of commarids and parameters that 
tell pitch, loudness, duration, attack, timbre, etc. He can edit ttie compo&itifm using the staruiard 0T5S editors. 
When ready he runs a program that converts his com|X>sition to information transmitted to the minicomputer 
driven symhesi/er. The souih! is heard m a few seconds via a startdard stereo system. 

Sev< ral surprising things hjve happened. First the course in composition is now attracting students with 
little or no ability as (i^rformers, since the computer is now the performer. Seosnd, a few students are exploring the 
idea of music as a list of data that can im transformed by means of al^ithfro that transpose pitch, invert pitches, 
change the sequence, etc. In short, they are doing music with the same eycitem^t as« and tetter tools than many 
professionals have. 

Tfie list of examples could be extended easily-river basin ec(»ystem simulations, economic simulation 
games^, computer experiments in genetics^, computer graphics as a medium for artists— but space afHl time do not 
permit. What all these examples have in common is a use of the computer in whidi the student mssWfS a computer- 
based system in order to do problems and projects m various fields. He is net nrterdy the slave of an auton^ted 
lesson, however personalizrd and with however much feedback. If my convictions on this r^int are not dear. I 
will conclude this section with a Chinese proverb: 

I hear, and I for^t; 
I see, and I remember; 
I do, and I urKferstand, 

The "Lfbrarv Model" Taken Seriously 

The claim that a basic facility tn computer use is every bit as necessary as reading and writing or arithmetic, 
has far-reachmq consequences For if basic literacy is not the exclusive property of university administrators, nor of 
researchers in the humanit»es divjstfjn^ nor of the department of linguistics, nor of the te^hing faculty, then for 



^Thr iMPPCIrSS Primer. 3fU e*l iPfOjac! IMPRESS, buft mouth Coilmga, HmnoM.NH, 1973K 
^Us*f n^anuatsare ava»1a*»le from J. ¥i. Appleton. Mutic D»pt.. Diwimoufti Coli«g». H»mji«T NM. 

^(j. S. Phlox, J W. So'Drt^Df. ' ACn£S A C«n*1 Development #nd EnvironrrMWiUil Gmmm'\ Un P. Pmttmnan, td., R*cent 
D^^^'lopm^jnt* m Ur<Mn C>^nnf\q, Simuiaf'ons Couf^d ProceetJtntfS. VoT 2, 1973). 

^ . R. C< '" i' >t*«f Mo.U'K <n ,%ts J ntrtnliif f Or y Couf*© »r» BKJfogv" Proc. Conf. on Comput«r« tn the i;n<1«r<}f 

Cuff.CuliJ. (CoFnin»<i»* f>f»t^?f, UoivWS»tv of lOWd. tO'V*»CitV. ^owv«. 19731. 
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the same reasons university computing must also not becxinie the exclusi . j captive of any of the analogous groups. 
Literate, competent and even creative use of computing ought to spread through most parts of the uniwrsity. 

The best institutionul mode) fur a coitifKiter center <iedtcated to this goal is the university library, which has 
always served the literary needs of ihe er^tire dckiemic commimity. 

The umversity library is an interestif^ object. Its director, the librarian, usually reports to the provc^t (or 
academic Vice President, or equivalent), who represents the broad acadeitfic interests of the institution. The 
librarian seeks advice and counsel not only of the Mnguisticaily based departntents, nor of the directors of 
research programs, but of menrtiers of the entire academic community. Typically there is a committee on libraries 
with faculty and sometimes student membership. The committee makes reaimmendations to the librarian and to 
his boss, the provost; atni it reports back to the faculty. In effect it aas as an academic watchdog on the 
accountability of the library. 

Access to the library is grants to all members of the university, as a rule. Some basic level of service is 
offered equally to ev^^yone, independent of rank or prestige or source of funds. Mere precious services, such as the 
rare-bocA collection, are limited by an assessment of the need of an individual to use tt, % not by his ability to 
pay. Lit>rary hours are determined by the demand of the dientete and not the convenience of the staff. Library 
use is not batch-processed. 

Funding is the critical factor. A university library is a cost center, not a profit-and-loss center. It is not 
expected to generate significant revenue. The fact that it is one.of the largest cost centers is justified on the value 
of the service it supplies to its varied users. Imagine what your university library would be like if its fiscal 
operation were modeled after the average computer center. Use of the library would be a pay-as-you^o basis. At 
my university the cost of the library, if distributed over the aver jfe number of books drcidated, amounts to about 
ten dollars per book borrowed. How mudi use of yoi^ library would you make if you had to pay ten dollars for 
every book you withdrew? Clearly, only those with substantial research grants could afford such use. And vyith a 
greatly shrunken user community, the cost per circulation might go up five or tenfold. Massive educational use of 
such an expensive service would be utterly unthinkable. In the end the type of library ^rvice available would be 
dictated by the interests of a handful of big research users and the department of library sden<%. It woukJ not be 
the library that you know. 

Though one has only to go back a few centuries to find it in practice, such a model *or administration of a 
university hbrary is heretical today. Paradoxicalfy, the application of tc^ay's library model to the administration 
of the computer center is viewed by most people as equally heretical. Yet in my opinion the logic that leads 
universities to support costly libraries available to ull must also lead them to support costly computer centers 
available to all. 

Dartmouth College is an exceptional university in this respect. Ten years ago Dartmouth decided that 
computing should t>e as accessible to students and faculty as a book on its open-shelf library. E^e of access meant 
the adoption of two radical ideas. First, a user's decision to use the OTmputer had to be divorced from questions of 
payment or formal justification. Second, the operating software had to be designed to serve a community of four 
thousand potential users. 

The result today is the Dartmouth Time-Sharing System (DTSS)^^o. In 1964 the first practical time-sharing 
system freely accessible to an academic community was created at Dartmouth. Two mathematics professors led a 
small sc|uadron of undergraduates m designing the time-sharing operating system and defining a new eaVy-to-learn 



F. Margraves. A. Stephenson, "Design Considerations for an Educational Tim©-Sharlng System". Proc, AFIPS SJCC 
1969 Vol, 34 pp. 657 664. 

'^J G. Kcmeny, T, E.. KutX/, " Di'f imou th Tirne-Shdf my**. Sctunce 162, 11 (1968)- 

^^Bionniai Report. <Klew>t CoiTipvitation Center, DarTmovjth College, Hanover. NH. 1973). 
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{»'€Kp'd*^n^ng lariguagi? - BASIC. The system could support the then unheard of maxinmm of 24 simultaneous 
mers. That system, which was designed around the Carters} Electric 2(M) oomfHiter series, (t^rcame the prototype 
for GE's Wlark I time sharing service, the first commercially successful time-faring service. 

Today DTSS operates 0)i a much lari^ and faster Honeywell G3S con^)uter. On this madiine DTSS now 
supports, measurably on any aftentoon, ovrer 140 simiritar^us users. Between one and two thomand different 
people use the Dartmouth computer in a day - about four thmisand In a month. Between fifteen and twenty-five 
thousand jobs are executed each day. About thirty institutions other than OaftnK>uth our comfHiter via a 
multiplex network that extends to New York, Boston. Chicago. Montreal and Toronto* Tfiere are about thrra 
hundred computer terminals on our campus and |)erhaps another hundred at remote tocations, OTSS comes on at 
7:(K) a.m. each weekday viful remains on for twenty hours, until 3:00 a.m. the next day - a toti^ of d hundr^ 
twenty hours each week. S^Jivice interruptions are rare — ^rhaps one a ^A^ek — a: id reasi^ry is rapid — usually 
under ten minutts. 

So much for system statistics. The upshot is that DT^ is a lar^. tt^vily used oomputCNr system. Adnunistra- 
tively, the computer center follows the library model quite closely. Both the librarian and the computer center 
director report to the Vice Presiilent for StiKtent Affairs, a fact which em{rft^2es our concern fw the educational 
program. <The comput<*r center director, Thomas E. Kurtz, w^ one of tho^ two mathematics profesors who 
originated the system. The other, John G. Kemeny, has left computing, thou^ he stIH a computer terminal in 
his office as President of Dartmouth College.) Like the CcHincil on Libraries, ther^ is also a Council on Computint) 
largely composed of faculty members. Like the library's public reading room, the computer center of^rates a 
public terminal room open to anycwie, with or without a Dartnwuth connection* In ^kfition it '"circulates" 
computer use around the campus via the ♦'egular telephone network. 

(Money, Budgets and All That 

"What about funding?" you are sure to ask. "How is it possible to just ^ve away such a costly service?" In 
fact we do not just give away tree computing. A substantial minority of the use of mjr system con^ from 
federally supporteil projects, and there is a law against charging the U.S. government for sonr^thing that is free 
to another customer. Therefore, we carry out a detailed billing on every user of the system, and w« have an 
auditable rate structure based on costs and aggregate usage. The individual student never sees a bill, however, and 
so to him the service appears free. The faculty member without support funds for computing likewise char^ his 
use to a ^neral unsupported teaching or unsupported research account. The bills for these unsupported accounts 
are paid in the end by transferring i^>neral funds of the College into them. 

The principal point to emphasize here is that computer use is not allocated by agreeing in mivance on a 
computing budget item for each academic departn^nt or other unit. We teliei^ that such a strategy, lA^ile it may 
appeal to a cost accountant, is simply wrong for educational €X>mputing. It ^sumes that the i^r of the resource 
and the allocator of the resource have a good idea of the probable value of eadi quantum of tse. This is true when 
allocating food m the cafeteria and so food should be charged and paid for by the meal. But is is not true of book- 
borrowing or computing. Worthless humdrum hours often precede |;»^icel^ sea>nds of imi^t. Value is hard to 
put in economic terms. 

On the other hand, any computer center has a finite resource, and an allocation problem must exist. In fact, 
while Dartmouth gives free access to the computer to anyone, we nevertheless ration use by other means, some of 
which are only possible with a large time-sharing system' ' . For example, we set a maximum run-time limit per job 
for each user. For students it is 32 CPU seconds per job. For faculty it is 64 seconds. We set a limit on number of 
characters of saved files on the disks. For students it is 8{X)0 characters. For faculty it is ^^CKX). (Total disk 
storage is 400,000,000 characters.) Access to system peripherals, such as the line fainter and card reader and 
pjnch is also rationed. We rio not place limits on total CPU use per session or per nwnth, nor do we limit the 
number of sessions at a terminal. 

^ 'j. S. McGeachiti. "A Flexit>l«? User VdlMiat»on Lan^uoqe for Tim^-Sbaring Systfims". Proc. AFiPSSJCC 1969 Vol, 34 
pp. 665-671. 

33 



Ttit? vatil majonty of our users rarely •jncounter their limits. Those that do f reqii^fUly f iml ways to make 
ttwH programs more efficient. In the end if all else fails a user Mn appeal to have any of his initial licnits 
Hii f ♦'aS4?d, As a yenerul rule his ret^uost is qfiinted. hut if he nmls still more, some supijor tiiif] evirieitce is ret|uinitf . 
{Stew ttmits are usually viewed as temporaiy adjustmetUs. 

With our ^resources and in our user community this system works. To the lieginning user the computer ^ems 
to tje a free rescHirce. The sense of freedom enables the user to approach computing on his own terms and with his 
own sense of values. It permits ^me-playing and trivia. At the same time it protects tfie serioi^ user against 
unrestrained use and abuse by the more frivolous user. In a lar^-scale time-sharing system, after att, one user's 
fnvolity can consume at most a few f^rcent of the resource. By contrast, one "big user" with a fat ^vernment 
grant can not only consurr>e a larger fraction of a batch system; he can also dominate the computer center ar^l 
distort its priorities, 

SMI, I have said nothing of the net cost of computing at Dartmouth. Isn't it vet y expensive to provide thiS 
type of <;ervice? Our view is that it cxKts about as much as it should. We geneiBlly agree that it is as imf/ortant to 
us to have n good computer center as it is to have a good library; Dartmouth bud^ts roughly the same amount for 
each. The total computer ranter budget ss about $1.5 million. This includes remal or depreciation of hardware, all 
staff salaries, heating ancf maintainir>g the building, and all other items attributatde to computing. Certain income 
choulcl he removed from that figure immediately, Since a qreat deal of our use is sold to non-Dartmouth users. 
Currently j^»out S300,0(K} comes from this source. The cost for Dartmouth use is therefore about $1.2 million. 
With 4000 students, that comes to an average of about S3(M) per student or about 9% of our annial tuition fee. 

This f ' jure is computed in the mMt conservative way possible and is the upper limit on our cost of comptiting. 
Note that it takes the cost of all computing at Dartmouth ami loads it on the student. The fact is that student use 
amounts to only 30% of total Dartmouth use. Therefore student iise costs only about Si 00 per enrolled student per 
year, which in 1964 dollars is exactly what the Pierce RefH)rt^^ recommended ten years ago for educational 
computing in the United States. 

To conclude this section, we believe that Dartmouth's exj^rience proves that computing can safely be made 
freely available to all members of a university at a current cobt that compares favorably with that of the library. 
Since computing is one of the few areas in education where co?:ts are going down, the situation in the future should 
actually improve. 

Proiect COMPUTE: a Tactic for Educational Chan^ 

Even though the mere act of installing a large free-access time sharing system ha^ by itself caused profound 
changes in education at Dartmouth, the rate of change has t>een slow and the impact on other universities, 
slight. 

Now I want to describe a modest prpject that might serve as an element in a national strategy for bringing 
about the sort of change I have been talking about. Project COMPUTE ^3 jg gn effort to develop and distribute 
courst^ materials dealing with computer use. The project is located at Dartmouth Collect and has three-year funding 
from the National Science Foundation. 

Publication and wide dissemination of approximately twenty textbooks or monographs ic the immediate goal 
of COMPUTE. The writing, testmg, technical documentation of, or dissemination of computer profira^ns is of 
comparatively little interest to COMPUTE. Cqually uninteresting to COMPUTE the problem of developing new 
strategies for us/ng the computer in education. 

^^Computcrs m Higher ttluc«tion, Prosulflnf'i Science AHvj^ory Cofnmttioe, Tfie Whif o House, Washington, DC, 1967. 
^^NSF Projott OJ-2S456. Informntion jhout the prOfect or for pf ospoctjv/*' Huthni^ »5 svmliihle from Project COMPUTE , 
Dart.Tiouth Collect, Hanover, NH. 
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instead, the pro|i?ct assumes that gotnl computer uses already exist and that technicat trafisportabiiity-i.e. 
moving prociranis to a new computer and suppiying operational d<K:umentaiion— is not a serious probtem. At le;»t, 
we think it «s not ttu? principal obstado to the sr^^ead of computer use. 

What is the pnnapal obstacle the shortage of good rnateriafs for use in courses where computer use makes 
SiMise. At the university level cHittcational cha*ige is almost always ^xomfrfished by the appearance of a new, 
commercially avattabte textbook. Change happens when facuHy • ,ffiiH>ers decicte to adopt rtie new book instead of 
the old one. There is no reason to believe that widesfw^ead computer use will h»open iu any other way. During the 
last five years hundretis of teachers have deveio;^ thousands of interesting apf^tcaiions of computing to their 
courses. Over two hundred pafM?rs on the tofiic have been liN^rited at four annual national mrotinsp SfK>nsored by 
NSF^^. Yet, for the most part, entluisiasm has becrn a>nfined to the originators and has not fc^n transmitted to 
colleagues. Interest has lieen ijenerated, but not sufficient enthusiasm to yield widesprem) imitatlOT. 

The reasons are simple. The amount of faculty time and effort that goes into development of computer uses 
is enormous. It is unreasonable to expect that kind of investment every time a coi»^e is ^ven. On the other hand, 
if teachers were aWe to get from tht'ir local textbook purveyor a few readable, f^ago^cally practical examples 
of course materials in their own disciplines, I believe that we would see an efKJrmous growth in educational omputer 
use. That is the working premise of Profect COWPUTE. 

Now a word atiout our operational strategy. On the basis of formal pro|:H3sais submitt^ to us and referred by 
others, we select six or etght people for summer writing projects. These authors leave their home institutions, 
friends, secretaries, telepliones, and in-baskets and come to the woods of New Hampshire for one or two months. 
The mission uf each Is to write as much of a monograph or text book as possible there and to ^ee to see it 
through to completion when they go home. 

For its part the project gives up to two months sunwner salary to each autfu^, and the staff attempts to 
create a stimulating climate for thinking and writing. Weekly Si st; ; allow eadi author to learn virhat the others 
are doing and to disaiss problems together. Access to the Dartmouth romputer is freely available to all, and we 
supf>ly a stuilent programming assistant for any author who needs or^. 

Much of the contact is informal and centers arouraj the computer center aiffee pot, corridor conversations, 
and weekend outings. 

Beyond direct salary support, our principal service is editorial. As each author finishes a draft of a chapter, 
the project editor types it into a computer file. From thence forward all chansps are made by modifying that 
original file, using built-in editorial features available on the computer system. Anothm^ computer-based system 
types out a matly formatted, paginatetl copy of the current state of the mantacript whene\^r requested to do so. 
The result is that even in our nine-week period of intense work, an author can see drafts of his work in progress, 
and can improve the quality of his product. 

At the end of summer, or more typically, six months later, the project has a completed manuscript from 
each person. Several options now exist. From the computer file it is easy to produ<» multilith meters for 
immediate use on a small scale by the author or others. One level higher, it is e^y in the same way to ^nerate 
automatically camera-ready copy with the quality equivalent to carbon-ribt>on typing. This makes possible an 
inexpensive format for large-scale printing. Finally, it is possible to go to automatic or manual typesetting and 
produce the standard sort of published hook. 

The role of the project at this stage is to act as author's agent in dealing with rommerdal fMjblishers and 
university presses. A unique and experimental feature of Project COMPUTE has to do with royalty payments. 
We are empowered to fiegottate the best terms with each publisher and then to negotiate separately with each 

^^PfOct*e<i}ng« of thi? con^cronrws, held at Unrv. of Iowa. Daftmouth, Atlanta, and the Clftremont COII0905. 1970 1073. are 
avuU^ifQ from th« Computef Center, Un*v»fSftv of lowo, lows City, Iowa, 
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author an appropriate division of the royalty J>etvveen the author and the project. We believe strongly, our 
authors agree, that royalty incentives are very important in any curriculuin deveioprnent project, since the regular 
academic rewords are so meager. 

Now I want to say a few words about why I believe that Project COMPUTE is a model for one element of a 
larger strategy for bringing about educational change. 

1 Change « produced by active te h&s with kieas that come dirwtty oat of their experience. The necessity 
to teach some subject is the moln »r of ttwir curricular invention, not the necessity to wite scholarly 
papers on educational change. 

2. Each author is tightly allied to his pmicuiar academic discir^me. Each is a member of an ordinary 
academic department. As a result each is an irmder and can carry out a program of chan^ in a way 
that outsiders cannot. Eadi will go back to his department and teach courses there. He may influence 
others at his own university; but, more likely, he will influence others in his own discif^ine at dozens or 
even hundreds of other universities. Communication within disciplines is far more profouraJ than com- 
munication within universities. 

3. Vw creative staff is a temporary staff. There are no f^rmanent irmovatian experts, only discipline 
experts who temporarily leave their conventional roles to ackiress a (»roUem of r^agogy. Before they 
fall prey to thinking of themselves as innovation experts we send ttiem home for more contact with the 
sources of their inspiration. 

4. Revvards for work in curriculum are not auj fused mth rewards for trmlitional academk gomi b^vior. 
Authors are paid to write, and if they write well further royalty benefits accrue. There is no pretense of 
enhancing professional presti^ or tenurability. Simple rules are the basis of gocKl behavior. We want 
our authors to behave like authors, not prima donnas. 

National Modd for Educational Ctiange 

A natural question is this: is it possible to extend the key fe?^tures of Project COMPUTE into a f uH-scale 
rational model for educational change and development? I believe that the answer is yes. 

The Carnegie Commission on Hi^er Education in a recent report^^ calls for the «tablishment of several 
regional centers focussing on uses of technology in hi^er education. The structure proposed there is the starting 
point for the full-scale model that I would like to promote. Briefly, each center is to be funded at a level of a 
small university, such as Dartmouth, for example. It is to have a f^manent staff of educators, witers and 
Sf^ialists in video, film, audio, graphical design, and computing. The long-range goal of each writer is to develop 
and distribute applications of technology that increase the quality or reduce the cost of education. Each center 
would provide educational computer service to its entire region. The centers would be situated in major 
metropolitan areas so that they would have atxess to real, live students in the way that the Open University d(MS 
in Great Britam. 

As far as it goes this is a good plan, but it does not go far enough. In particular, the relationship between 
these essentially research and development centers and the production oriented conventional universities is not 
dear. At some points in the Carnegie report the centers appear as service organizations for the universities. At 
others, they seem to be competitors, having their own staff and their own student clientele. 

It is here where I believe that our Project COMPUTE experience can be of help. Suppose that each center had, 
in addition to the regional tie suggested by the Carnegie Commission, a very strong dixiplirw tie as well. That is, 
each center would be targeted to produce applications in a specialized group of closely related academic fields. 

^^The Fourth Revolution Instructional Technology in Higher Education- (McGraw Hill, New York. T972). 
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Persons representmg those disctpimes, f^of^ibf y via the professional societies, w perh^ through editoriaf t)oards 
of t:*e respective journats, would serve as an advisory panel or even a policy revie\^ board for thecorf^fxinding 
center. Such a ()roup mnjht advise the center on iteeits, fmorities, and market^lity for new ediK^tioiiaf 
developments in their disciphnes. It miyht dho serve as an insider i^roup for the proimtion within the disciplines of 
new developments coming out of the centers. 

The actual development process in the centers would a partnership involving people with technology and 
media st^cialf^ation, education theorists, evaluators, economists, and, finally, disciplir» experts with teaching 
experience. The latter shook! usually be the senior partner, in my opinion; the others should serve as r^ource 
people. For the reasons stated in describing Project COMPUTE, it is better ttiat the discipline specialists be cast as 
''Visiting Fellows" rather tftan "Tenure Faculty". In tl^ first place, a permanent staff of innovates is hard to 
imagine. Second, the discipline tic of the permanent innovator would become thin if not refrfenished by renewed 
contact with peers in the discipline. Third, the number of people in direcrt contact with the center and its work 
would be vastly lar^ if the number of Visiting Fellows was large. Fourth, the return of visitors to their home 
universities would serve to spread the w^rd and market the prcKlucts of the center. Fifth, the rr^re possibility of 
becoming a Fellow at the center would cause many more i^ople to become conscious of and interested in the 
activities of the center. 

Since, in the conventional university at least, the effective consumer of educational change is not the sti^ent 
but the teacher who selects course format and materials, it is crucial that any large educational dewlopment centers 
have powerful ties With the teaching faculti^. That means, in our \^rticdily partitioned universities, (K>w^ul ties 
with academic disciplines. 

It might be argued that educatiotial change cannot happen within universities any way, so why worry about 
good relations with them? Why not set up centers that will compete with them in doing the job that they now do? 

I would hope to see a healthy element of competition between the centers and the regular universities. 
The presence of "Open University" students is a good idea and would foster that competition. But there are two 
reasons not to cut off the ordinary universities from the start. First tfiere ijs no need to. The model I have 
proposed poses little threat and even offers opportunity to the academic disciplines. Swond, I seriously doubt 
whether a frankly competitive organization couW attract a staff of sufficient competence to win in a struggle with 
the conventional universities. It would \ye att^ked by academic specialists as an iilegitimate upstart, and that would 
deter most of the best people from taking a job at the center. 

In summary, then, 1 propose several large regional curriculum developrr^nt centers, each with very strong 
discipline ties. These ties would be achieved by being held accountable to representatives of the disciplines in 
exchan^ for fegitimacy in their eyes. Apart from the question of ties to academic discij:rfines, the remainder of 
the model is essentially that proposed by the Carnegie Commission for developing uses of instructional tec!mology. 
Since that proposal is probably well known, I shan't elaborate it here. 

Meanwhile, Back at the Closed Umversity 

Unless someone in the audience knows something I don't know, it would be poor judgem ent to do nothing 
until massive support becomes available for such national centers. One works with what is at hand, and it appears 
that traditional, closed-wall universities are apt to be at hand for a while. Yet all is not gloomy thore. 

Dartmouth College has made a recent commitment of resources and personnel toward educational dian^ 
within a conventional university environment. Operating on the premise that the best source of ideas abo;it educa- 
tion is the regular teaching faculty and the best testbed is found in courses offered in the regular departments, 
Dartmouth has defined a new office charged with promoting faculty experiments in teaching. Particular emphasis is 
to be placed on experimtrnts that will lead to qualitative or quantitative increases in productivity through use 
of technology. 
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In view of the v^l kmjwn professicmal and political probleim faced by edijcationHnninded faculty members 
in regular departments, one of the principal functions of the new office is to ^rw the interests of such people. 
This will be accomplfshed in several ways. The first arid most important way is or^nizational. The director of the 
new office-^names and titles are not yet settled, but for |»^esent purpose let me call him Dean of Instruction- 
reports directly to tf« Ctean of Faculty, He thus has the ear of the university officer most directly omcerned with 
the quality of the faculty and with departmental practices and standards for hiring, promotimi, md tenure 
recommendations. Hence an organizational mechanism exists for influencing ttwe practices when they are in 
conf Jict with the purposes of the office of instruction. CXiviously. this mechanism must exercised with con- 
siderable skiil and political wisdom if it is to be eff^ive. But the point is that it mv^existSL Thar^ is a way for a 
faculty member to hold the university to account for its claim that irmovative teaching is rewarded. 

But accountability applies in both directions. Teachers who expect the support of the Dean of Instruction 
must be held to account for the quality and effectiven^ of their work, Fw this re^^on the Office of Instruction is 
charged with res|x>nsiWlitv for evaluating new ^ucationai project A mere wiitingness to experiment is no 
guarantee of quality or effectiveness; and, if only to jM-otect his credibility^ it is e^ntial that the C^n of 
instruction distinguish betv^^n good work and bad work before making a teoommendation to the Dean of Faculty 
te^rding particular faculty members. In this way faculty who work with the Office of Instruction wit) be under 
the same kind of tension experienced in a regular departn^ent, where they expect supfx>rt if they do well and not 
otherwise. 

Evaluation, then, is the second way in which the new office will promote effective experiments in education. 
The thifd contribution is technical and informational. Physical re^rc^, including standard audio-visual equip- 
ment, are being expanded and the staff increased* The campus will be linked with TV caWe and more ori^nation 
and playback equipment will be added. InexperienmJ faculty members will find technical assistant. 

The fourth activity of the Office of Instruction is to coordirate its educational goals with the purposes of 
the computer center* Not surprisingly the majority of educational technology proj^ts at Dartmouth have been and 
are likely to continue to be based on some aspect of computer use. HerH;e it is essential that, for example, a 
proposed project be examined for acMitionat demand it will place on computer resources. Fortunately, the primary 
mission of the Dartmouth computer center is to serve educational needs; arKl there is no conflict of fwrpose with 
the Office of Instruction. {One wonders how many other universities can make that claim.) 

The final role to be played by the new office is that of Fund raiser, it is currently assisting faculty members in 
four different departments in gaining foundation support for sf^fic projects dealing with educational technolc^y. 
Its success in this regard can be expmed to give the Office of Instruction considerable politial leverage when the 
time comes to speak out for faculty members who have carried out projects under its auspicies. 

While it is premature to report results for the Office of Instruction, I believe that the basic organizational 
structure is sound. It is tl^ test W3y that we can think of to promote experimentation in education within a 
traditional university, using regular students and faculty with expertise in regular academic disciplines. If 
successful it will not be as an isolated institute for educational research, but as a practical model for fostering 
and managing educational chan^ across an entire university. 
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THE AUOrO TUTORIAL SYSTEM OF INSTRUCTION 



Robert N. Hurst 
Profes^r of Bioiogy 
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i 

Let me begifi with a rather pointed statement which appedr-ti in another I prepared on the Audio- 
Tutorial System of instruction, "Neither its strongest suiD^orier nor its severest critic could cteny ^ impact of the 
Aiuiio- Tutorial System of instruction on education/' (Hurst, 1973). Sjfu» tte birth in 1961, A-T has indeed 
exerted an influence tKi educational thought and practices; it h^ indeed garnered \n shme of ctevotees and critics. 
Most assuretJIy, it has c<^tributed enormously to the inter^t in and unrest resulting frcmi tiie 'fourth revolution" 
described by Eric Ashby (1967) and canonimi by the Carnegie Commission of Higher Education {1972). 

We are in the midst of revolution in education and it is an exhilarating srui exciting time to be involved with 
technology, tsadiing, learning, people and students - who ^ after all, *'a tot like peiH>te'' ^fN^iethwait 1972). 
Those beautiful, bri^t ctemaruiing souls, eager, yet hesitant to le^-n, tired of traditiw, daring ir^tructors to 
teach them, ft is a time foi innovation, for dian^. In fact, in many educationiri settings, innovation and change 
are being legislated by well-meaning but tgrior int trustees, txsards and iKfministrat<^ who do not tmcfefstand that 
change cannot be legislated but must grow as did Audio-Tutorial fri»n an instructw's c(»iGern for his students and 
how they learn. To place the unconcerned or the doubter into ^ "innovatjve" setting and tell him ''now be 
innovative" is to assure failure. So many must be permitted, in spite of the knowledge explosion^ to disregard 
biblical admonition and continue to put new wine into old skins. The dd skins being^ of course, the lecture-lab or 
lecture-lab- recitation format, the '*$pray and pray" system where the professor as the font of knowledge Sfwws 
fcKth while each and every eager learner dutifully copies, swallows, digests wd tfien upon commaiK), regur^tates. 

But for the rest of us, there arc other possibiUti^ that make new promises. This is where tfie Audio-Tutorial 
System of instruction makes its debut as only one of a whcrie hc»t of possibilities. As does every other innovative 
educational practice. Audio- Tutorial has only one goal or purp«e 'To H^p SUf<^t^ Z.flam.'^ We all realize of 
course, that as facilitators of learning, or teachers, or educators or whatever title we take upcm ourselves, helping 
students learn is the whole purpose for our existence. If do not help students learn, we ou^t fK>t to be in the 
busKiess of education. Therefore, allow me to preface my remarks aiKHJt the Audio-Tutorial System with swne 
concerns we should all have about tedmology -oriented instructional systems be they A-T, CAI, video or some 
combination of these. 

Technology must not be used tor thj sake or novelt>' of the technology itself. It must extend, not former 
restrict our educational flexibility. It must enhance not cramp our edut:ationai style. It must help students learn 
not entertain, humor or occupy them. It must, on other worck, have a positive effect on learning. If it meets this 
final criterion, perhaps the use of that technology is justified tecause as Eric Ashby has said ''Any technolc^ 
vWiidi increases the rate of I'^arning would enable <as Comenious put it c^turies ago) the teacher to teach less, and 
the learner to learn more" (1967. p iii). Yet, it is essential that we examine a te<*inology, weigh its costs and its 
benefits to ascertain its value as an educational tool now and its potential value ten or twenty years from now. 
Technology must be kept in its proper perspective and as indicated by the Carnegie Commt^ion, "it should not be 
adopted merely because »t exists, or because an institution fears that it will be left behind the parade of progress 
without it" (1972, pJI). 
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The Carnegie Coftimissiufi in its Fourth Revolution report (1972, p. 1! ) lists two tests which ou^t to be 
applied to determine whether technology should be used in an instructional program. Let me cmnmand them to 
vow. First, the learning task to be performed should be essential to the course of instruction and second, the task 
should be performeif better with the use of technology. Perhaps the task could not be performed at alt without 
technology. If so, then only test one need be applied and the instructor has his answer. 

Technology is having its effect on the educational process and the field of technology promises bigger and 
better things yet to come. Communication satellites, video cassettes, miniature computers and cable television are 
iust a few of the new tools promised and in some instances already delivered. 



This special fssue of electronics magazine is illustrative of the directions some tools are taking, A special 
report of tomorrow's communiciitions. 

No one would deny we are indeed a tuned-tn, wired up, pluggcd-in, hyperarticulate speed of ti^t society. 
And tomorrow's education may well start with today's design. 

We could have no more pencils, no more books, and we could write and read electronically. 

The goals: A communications system that replaces person to person contact. The tools: Satellites, fac- 
similes, computers, telephones, and microwave. This sounds exciting on the surface, doesn't it? Yet one has to 
ponder whether or not this is the direction we want to move. What are the roles of or the limitations of 
instructional media? Do we really want to replace person- to person contact with all of those very elaborate tools? 
I don't know; maybe we do. 



Electronics 
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Perhaps however, the most important point of this particular report was a comment that appeared near the 
end, where Albert Einstein was quoted as having said that the container would never be more valuable than its 
contents, I submit that media^ that technology must be scrutinized carefully and their value documented, else the 
container may well become more valuable than its contents. If a computer program is designed to operate as an 
electronic page turner, I'm not sure it is really worth the money. The container becomes more valuable than its 
contents. At Purdue we program students to turn their own pages. Do we really want to replace this person to-person 
contact? 
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in the final analysis we are still dealing with peopfe, studfsnts like this typicai, average ruivof-the mill pre>vet 
student. I for uiit?, am not ready to give up meeting this student on a one-to-cwc tests in a person-fo-person contact. 

But wfiere is this talk alKiut xUv inv aiuJ pusstble misuse of technology leading to? After all, my mission is to 
inform you of a technology-based system of instructt<Ki called AudtchTutoriai. My assignment is not to bias or 
prejudice your thinking m terms of what you hear from others during this symfK^ium and I'm usm i have not, i 
do have re^ concerns about how and why technology is used as I know the other fKcsentors do. These roncems 
lead me to t^lieve that the best way to help my students learn is thrcugh the tise of Audio-Tutorial System, 
just as the concerns of other innovators dictate the use of their systems to help their students tram. 

Perhaps a discussion of i^te mo<ies and media of the Audio-Tutorial System should begin with a history of its 
evolution, t^cause like a viable living organism, it evolved. 

The entire program grew and developed because a single individual Dr. S. N. fH»stletfiwait of the Department 
of Biological Sciences at Purdue University, was concerned for his students and their varying ^ilities. Of, Postlethwait 
had for many years been involved with introductory course for beginning freshman stuctents ami for many years 
had not been completely satisfied with the lecture-laboratory technique of prestation. It was obvious to him that 
thou^ he was an excellent lecturer, {we are all excellent lecturers of a>urse) he was not reaching all of his students. 
The course he was teaching encompassed a broad spectrum of ability levels from the very low level achievers through 
the very bright pre-veterinary students. Dr. Postlethwait knew he w^ talking over the heads of n^ny of tiiose at the 
low end of the spectrum and at the same time he was not stimulating tho^ at ^ hi^ end. 

In order to better meet the needs of the individual student, he b^n to experiment by preparing a tutorial on 
tape, (^k>t a lecture on tape, contrary to many people's beliefs J He prepared a tutorial — a one to one discussion — 
as if the student were seated on the other end of a log from his or at his side or on the otfier side of the table. 

He made the taped tutorial sessions available to students in the Audio- Visual Center whk^ was open from 
7:30 in the morning until 10:30 at ni^t. He then announced to the st idents in his i^mire session that if they 
had difficulty in keeping up with him or it they found themselves not bsing stimulated by his lectures, they could 
go to the Audio- Visual Center, check out a tape and listen %p the weekS maferiid as it came to thmn in a different 
format. They could individually pace themselves, by deciding just hi^ much of S- N. Postlethwait tfiey wanted to 
listen to in any given period of time, by repeating those portions thjt tfiey didn't umierstand the first time, by taking 
notes if they cared to, by isolating themselves in a booth with he^ sets from alt extraneous rwise and prinwrily by 
deciding when they wanted to learn botany. 

Being the sage that he is, Dr. Postlethwait soon realized that he had a very important tool at his hand. It gave 
him the opportunity to involve the student with the thin^ of his discipline, with plants. He couldn't do this in a 
lecture of five hundred students, but it was quite possible to carry some plants over to the Audio-Visual Center and 
put them in the booth with the students. He a>uld actually tutor him now through botany by manipulation of text 
materials, laboratory manual and short exercises that could be done in the booth by involving the student with the 
tangible items of the discipline. It was a true multi-media approach. 

A very peculiar thing happened; students stopped coming to his lectures. More and more students d«:ided that 
this kind of»format had something to offer. It was nice to be able to go to the Audio-Visual Center at 7:30 in the 
morning or 8:30 in the evening, anytime they wished, dieck out a ta|^ and be prc^rammed through a series of 
learning activities. 

We can conceptualize the Audio Tutorial System perhaps with the diagram now on the so'een. If we know what 
it is we want the student to be able to do at the end of his peri<xJ . i instruction, we should tell him what is expected 
of him. The numbers 1 through 4 on the ri^t side of the diayam, signify the j^rformance objectives which are 
spelled out precisely for the student. The lettsrs A,B.C,D,E,F signify learning activities which will help the student 
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ABC D E F 



1 
2 
3 
4 



SZ/cfefi 



mrm* ttio pf*i f Of mance level desii^iated by thtj objectives. These activities may include some reading ffufii the 
ftudy (|ui(ie, some demonstrate materials, CKamuiinga slide or live specimen, viewinn i sequence of film oi doing 
an experunent. 



They w^vy lmt> \AWdvifiy A,B,C,0,E dod F, touether signifies the audio-tape which is simply the vehicle 
whereby one f>fogfams the student throtjgh activities A, B, C, E and F. It is not a lecture on tape; it is never a 
|j?cture on tape, but it is a device, a firogramming device, to lead students, quite lorficdlty from one activity to 
another, one experience to another, to get him to the performance level specified by objectives 1 through 4. The 
taf>c !S not the most Important aspect of the program, the objectives and the activities are. Individuals who do not 
have a sound grasp of the Audio-Tutorial philosophy, fail to comprehend the meaning of those words. Any tech- 
nolociy which could deliver this unifying thread that ties activities together could be plugged into the system, 
video, film or a combination of these. 

Why audio tape, why not video or film or print? Audio tape is a now technology, a low technology. It is 
extremely reliable, inexpensive, tlurable and available /jow. It lends itself to the tutorial aspect of teaching, which 
means a convefsational presentation not a formal lecture, which also means individual pacing not group participa- 
tion. This also fneans that the student may stop, back up, and replay any portion as often as needed for under- 
standing. These attributes are difficult to achieve with video or film and too expet)sive for most schools to achieve 
with the computer. When the technology of any or all of these formats permits the same flexibility at the audio 
iaf>f» at a comparable cost, then indivitiuals like myself would lie willing to t)ecome adv(K:ates of video-tutorial or 
comfXitef-tutorial. What I am trying to impress upon you is the tutorial aspect of the program and to indicate to 
you the audio tape is simply the most convenient, inexpensive, reliable technology available now with which to 
function. 

Hopefully you are asking yourself, "Why technology, why not print?" Print works; we know it does. It has 
servetl the bndgt* arnonq scholars ^nd bi'tween scholars and students for centuries and it does work. Just about 
everyt^me I convince myst?lf that it ouyht to work as well as the ^udio tape, I find something like this in the 
newspaper. 
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Recording 
An Increase 

U3(ND0N (UPI)— Gordon 
Jones, concerned because 
i^any of the p(«:ential cus- 
tomers who drive into his 
self-service gas station 
drive off again cm reading 
the instructions, installed a 
tape recorder to explain 
how to (H^erate the iHimps. 
Business shot up 300 per 
cent, he said. 

SMe8 

So w8 are an aural beast as welt as visual. 
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But it really isn't just to titilate the aural sense that the audio tape is used, another real function comes in 
freeing the hands and eyes to integrate experiences as this student is doing - n^niputattngd tangible item and 
Study guide while listening to her instructor talk to her in a one-to-one discussion. What would you do if you had 
lie student to teach? Would you sit her in a lecture rocwn and lecture for fifty minutes three times a week, then 
have her come into your lab on Ti^day at 1:^ for a Three hour lab v^A^re you would attempt to integral what 
you had told her in the lecture with some tab experieni^s? I think not. You would probably try tb arrange scwne 
equally convenient times with no set definite fcroundrtes and tutor her tiirmjg^ a series of learning activities which 
would include laboratory experience as the time they vrauld be most m^ningful rnlhrtearning ^equen^. You 
(:^obabiy would not tell her about cell structure on Monday and Wednesday and have her rome in the follcwing 
Tuesday at 1 : 30 to examine some cells under the microscof^* Well if wouhJ not l^ure to one student^ why do 
it for 1000? If we can tutor one student, can we not tutc^- lOOO? \ 




Obviousiy, we cannot as a single individual sit on the other^end of a log and tutor 1(K)0 students throu^ a 
concept in biology. But we can place a lafx recorder on the other end of that log with a recorded tutorial which 
will tutor him through a sei ies of learning activities — a one-to-one conversation with the student using ail the 
available media ax our di$posal. What media are available to us as educational inputs? 
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Really there are only four vnapr kfjuls ut imputs th<? printed, the physical or tangible item or model, pro- 
jected materials and aud^o. Thf developer in the Audto-Tutorial system determines some real objective then sur- 
rounds the student with any of all of these mpuTs an<l tutots him using these media to reach the objectives set. Any 
{|iven mstructfonal packat^? may utili/e one or all touf of these inputs depending on the objectives, the availability 
of the medium, the erkicational value of a given medium in a given instructional package and cc^t. We would not 
project a four minute &nm film wlwn three color slitk^s mic^t be as eff^tiw. We should not use a set of slides 
when a pa^ or two of line diagrams in the piintetl stutly guide wouM probably do the j<rt) welL On the other 
hand, if motion is an integral factor for illustrating a given procedurc/thc student should have motion available to 
hnn. In our sttikition, this would be 8mm film. 

A T involves the student with an audio mput. 
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and with printed materials 
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when projected materials are approfM^iate, 
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And it involve , the student with ljn<jibtt' items, 






TANGIBLE 
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when tjnyible items are appropriate. 
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My chofco of the word mvolvo in the past few sentences was not accident^.:, for this is a major responsibility 
of thf dfiveloper of dn Audio- Tutoridl instructional package - to involve the student with the thiogs of the discfp- 
fine. Immerse him jn botany and things botanical if your intentions are to teach him botany. Have him wrapped up 
in all the mwiia appropriate to the learning task. In fact, it is my contention that any "new and excitmg" program 
wn think to involve ourselves with whf?n wt: attempt to innovate, ought to have built into it the potential for using 
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as nuny of thesf* avuilai>it? inputs as possible. No systt fii sluuHf i vvut k ti» thu «i)<clusion uf afiy om? of thuse tnstruc- 
Uoujl mputs if tht*y incU»«Mi have any pufinttujl vjlut* 

So vvt; burfound the stiulrnT wilfi ali iUx'sv n^Miuj m a Lo.jttiiMii Center wtwic he cijinbe himself, take off his 

shot's 



fotax 
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and bocomo invulvcii with the U'atnimi activities. 
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The stuilf'nt necuint's isoljtfii fiufn extraneous noisf, procewls at his o» ht'i own rate and yet is able to get help 
frt»ni dfi ifistructfM wtui in .ilways jvail.diU; 
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for immediata and personal help when needeii. 
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as in the case of this student who has sattstdctonlv completed an instructional package we call a minicourse and is 
now helping a student having difficulty with the same minlcourse. The student will receive points which c^n con 
tribute to a hiq^ier yrade in the course through this teaching activity. 
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Or exchanges like these taking place informally in the coffee and waitinq room adjacent to the l.earnjng Center. 
Many exchanges here are related to biology, but we are not <;ertain how many are r^'lated directly to the course. 

Well those are the modes and meciia of AT. now whj^t are we doing with thi*; «:v>tem that capitalizes on the 
flexibility and the chance to experiment with learning that «t affords us? Can we /cro <n on some of the things we 
are doing with the system that we cannot kk) with u IraditicHi^j format? 

In 1969 we decided it was time to try some other new exciting irteus and we introduced the concept of 
learning for mastery into our Audio Tutorial program. This particular concept opens a whole new field of pos- 
sibilities. The opportunity for one thing of making time, not performance the variable. One can set a performance 
standard at the level desired, then make time the variable in allowing the student to reach that minimal level. 
What happens in the process is thai students are not branded failures, they are not told "you did not or cannot 
succeed." Sack m 1963 a gentleman named Carroll pointed out that this attribute we call aptitude which is apparently 
so important to so many people is really nothing more than a measure of time, "the time it takes a student to learn 
a particular task/' (Carroll, 1963). 

Benjamin Bloom (1968) points oiit m his rUtK^le "Learning for Mastery" that one thing we can do when we 
employ this particular concept, is document as some measurable minimal level of performance, that our students 
can at least accomplish certain physical or mental tasks. In so many teachmg situations this is not po5Sible and 
grades are meaningless. I wouln't want to go to a surgeon for an appendectomy who had A's in vasectomy and 
tonsilectomy but who had failed appiMidectomy. His overall grade might be a C m surgery, but I would piefer a 
learning for mastery format so I could know that at least at one time, hf could do an appendectomy satisfactorily. 
Obviously, this is a grossly contrived example, but when one employs the concept for learning for mastery with 
precisely-stated objectives, it is easy to document for anybody who cares to examine the program precisely what 
it IS students c:an do. Stu(ients do not completi^ an instructional unit until they reach a minimal acceptable per- 
formance levrl. 
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A learning fr" ft^dstery format rtniJiHcs a i.rfiul mstfuctiofial fwckdfj*' so . studmt does not reptMt compl»'f*? 
units of material v^uen pt f haps only on«' or two points are nf>t well understCMid t first time through the in .tructiun. 
At Pufdut! Univ<»rsay w<» use ar^ instruct «Mnal p.ir;*,i<|* vve call a rninicourse, whi< '\ is a unit of instruction thai takos 
the avotaqt> student afiywhi»ri» from Vj fipuf tut ^inv fnimcour and up t«i 3 fu.urs fur other minicour*.es to 
complete, dependinq on the prrnriple or concept inung discussed. 

Minicourses are, just as thnr juimu unpl*^ ^ Httle cours4»s. Each has a heiiinnitiq and an end ami eai.h relates to 
aniither in virtually the same patt»;rri as one cun se in any collegt* curriculum relates to another. Thus some have 
pterequisites. others <ki no? ^» jme catty ntore C(*Hlit and so are more u-nse or longer than othets. Just as some 
Cfiufses involve labs, field tr<ps ui Jemorjsti,.;. >ns or as som*? are more visual than others, minicourses tnay vary one 
from another 

MiniriMirse content is perhaps mc? ^tnjiral th.u, the conteiu to some courses (which may be artifically inclu- 
sive of SfHH:ific topics within » t|iven discipline), Minicour«;p*; are d« ^ujnetl around a single topic or idea or principle. 
1hese divisions are much more natural. 

Upi)n rompletion itf »i minicourse„tnr student rec t ives 'mmmhaTe feefib^k in an oral and written tjuiz on 
hi. fierfofniance m th<H particular ins^ructiunal iwckaqe and s allowed to move on to the next if he <s successful. 
Some stu k'nts do better if cjiven fret|ti"nt fe »di)ack of sue f > .s and this system has great jiotential as a motivatinq 
factor for that kind of stuiient. If a sttjdont docs not do well Uie hrst time, this tells him something ,\\iout his study 
habits .i>mething he fines f)ul havr to wait until the en » Mf th*^ semester to find out. It also tells him something 
about his won abilities, f^or example, it nuiy' not tie ei-oM |h for him to go through the material only one time. It 
gives hir a chai^ce tr> j^'t started in the right direction in a course and to work at some of these *Mrly instructional 
f>acka<u ' until he achi»»ves the sturly proceilure nt-ed* fof to succeed. 

At the same tinif, this kind of course will *.eT as a windowing agent for those students not capable of com- 
pietmij rhe program. Bloom (19hH) {>*Hnts out that rhrnev^'r the pain of remain in a program tjecomes unbearable 
to the student, he sunfily vfill quit the procjram Sr> the winnowing factor would probably st«il be present, but it 
wouL: t)e ttte student's decision lot t<i remain m t tpvei* curriculum - the student decides "»t's not for me/' he's 
never told "you're not quite up to our standards' which does nothing for the student or the school. 

I'm sure tij- idea of m.iking time a vanahlt' ranklf?s many indtvidt^iij involved in professional corrirula who 
feel there's somethin(| sacred atK>ul a tune seqnr'ncf^. c*' * a * mester or about a year. But let me pose this 
c^uestion: If we i >m designate what terminal i haviurs or what terminal kinds of performance we expect from tht: 
graduates of any kind of in*;^^ uctional pruotam, (v'h^ thei it be a course, or a bnr calaurtjate degree or a professii>nal 
1 ijree) are ive not better off if we can assur uosv. who use Uie services uf the graduates that those individuals 
can I'^rform at some sp^ cifit cJ level rather than stilting iu.ti they comtilet^'rl tfiii. prograi ' In a semester, year or a 
p' nod of anv designated length 7 

Thi.' courses in which I function are a botany course, Biology IU8, .ifid zoology course. Biology 109 When 
we converted to the C(iru ef)t of learning for rn r,tery on 1969 and looked at the content of these two courses 
which were always considered sequential courses but taught by two different nuiiuiduals, we discovered a very 
int* lesting phenomenon. Apparently nefther uistructor trusted the othrn cumplptnly and both instructors were 
tt a< hing their students several principle s common to both plants and animals ''students found themselves repeating 
much of tlie same material and thus, there was the problem of redundancy. 

Wirh the cor^cept of learnin(| for mastery, when you can assure your^flf that there students (u^ntMjt move on 
until they reach some level of proficHmcy with a rjiven topic, it is possible to eliminate all redufidancy. In 1969, 
w*' actually eliminated about one fourth of the redundant content of theve two courses and this gave us |ust what 
thf> Carnegie Comtnission {19/ I ) oromised in \hv\r n.'fjort Less Thne-Mon Options. In reality, our program f>re- 
dated the Ct)mmissfon's report hut our fuuJings suf)ported their contentuvis. We had some time left over. We had 
some optiofKii time to let the student pursue nome topics in tnolofjy which were of intf^rest or concern to hni] 
while working m the sam(» seme .ter rredit fiour framework. 
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Ifi ttifif stjcuful i:iMir.t» ot ilh stu|ueiK:t\ ^tudt'fits cIh)osi? ahout 40v';, of their curnculum from a whole host of 
opttoiuii min«i:uuf mjny of which were fvolviMl upon ii\ttn\l requt-st from the studt»nts. teli us what kind 
ot {KoKhju'jl T.»jHi''. Thry vvjfit to ^tudy, w<? dtrvelop <i mimcuufbt; make *t availiible to them. For special interest 
cfuMips. UK ' ttu' pr^' vf?l sttidf fifs whc^ wjnft'ti nunr jnjtomy hcca.isr tht^y WdfUetl (o better prepare themselves for 
theif ^idniiSMi'.n , xjiinfuiiunb, we luid lUr vrtiu;li- f** mukr this fn^ssihle. We havi' also develop«1 minicourses upon 
rtfc|uesl from professors uf rourses tjiHjhf bUhsfM|tjifnt tu ours tn majur areas. Often these professors have served 
«is con.,uit*jnts in the f jrep.jr jtron or h,«ve T;ikrfi jn .ictive developmental role with us. 

With a smalkf pat:k«KH? of instruotH>ndl fUJt^M mIs, one jIso jfUrcuiuces a flexibility thai «s not possible in a 
iradilinrul lertuft-lali situation. Just as cuthnip stiideiits ui the sttll remembermi past were offered more mdividu.il 
mni cnrru'ufa >t,\wn a vanity itf cout sob tef»Lu:<? the smi^le rurncuhim all students were expt^cted to follow, so lot), 
do miniroirscs further jndivutualt'e fhf* tui f h:uliif!i. The development cd the maiiaiurse or conceptual pai ka<p> 
made tt possible for us to irulivuiuallv prescrji^e msliuction for any stmlent for any option within the School id 
Agrit iilture and the School of Pharmacy which are the Schools serviced by our two courts. 
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We now have a bank of these mini{:niirsi>s from which we are able to individually prescribe a course in biology 
suited to the need of anv sper^^ic mciivuiiial or option. 

Students with qiiicjance cai^ iniild thejr own course. We ran offer vai lable semester credit hours from 0.1 of a 
credit up to 10 or 12 semester credit hours. Tins year we tire offering either 1, 2, 3 or 4 semester aedit hours in 
either course and we individually couns**! ,\ siudent into a package of minicourses commensurate with needs, 
interests and credit hour enrollment. Onr of the most surf)rismn aspects of uur new variable credit hour offering in 
just how many students in home pronorTucs or :,ocioloiiy or whatever, would like to have 1 or 2 cn^dW hours of 
biology. 

Consider also the disadvantaqed studt^rit. Spef:iftc mmicoufses designed to iflim.natc specific (ieficiencies make 
moreservse than enrollment in .)n {jntuo course* wh«'n the dpfirM>ncy may not be nearly as broad as the course con- 
tent. Instea<l of giving The patifnt tnir of pjcfi jnll ofi fhi shelf ti^ cure tfu? concldion, diagnosis followeci by prrs 
cnption uf fewere specific pills, r)Ossibiy fven unu, is th(f treatment r«M^uiri'd. Instructional packagi'S likf x\w nufu 
course Ctm function m this cap/city. 

What about the hcMist?wiff' m tlie htjsin^'ss man ui ttit' farmer who wouUi like one Sf?mestcr credit hour in 
biology, or who would lik^^ fu kn<rv "son^rtfnrKj" (ifMirtK S. f)r fninf?ral riutntion or phoTosynth^'si*. n\ birth 
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control? The minicourse format permits that kind of student access to specific areas of interest or need without 
the necessity of an enrollment in and completfon of an entire course. The potential then in rontinuing or 
extension or adult education is unlimited. 

What about the management of such a wide open system of instructions; is it impossible? No, but it is 
beginning to tax my graduate students and secretarial staff. Open lab hours, a cafeteria of learning which any 
week could potentially offer students about 80 different minicourses, but more practically offers about a dozen, 
a student body enrolled for differing credit hour values, each pursuing a package of minicourses perhaps unique 
to himself, all contribute to the management problem. We have long been ready for computer assistance and in 
fact have programs written or available to manage the system for us, but alas we find no federal, state or private 
agency with 2 to 5300,000 to put into computer managed instruction. 

The computer is destined to become the most important factor in any major innovative instructional under- 
taking, if not as an integral part of the instructional process, then as the manager of student records and testing, 
willing to deliver progress reports, prognoses and diagnoses. The present problem with obtaining computer assistance 
is the vast federal funding for some major CAI programs and the lack of any kind of funding for the mundane com- 
puter management tasks our own program now stands m need of. The real problem will arise when further educa- 
tional experimentation, which is sorely needed, will be curtailed by the lack of computer assistance in many 
innovative programs such as our own. 

Managtment problems increase with each new venture as they have this year by our movement into an 
experiment with residence hall learning. For two year we have had a take-home kit for all those minicourses whose 
tangible items lend themselves to packaging. Beginning this year we are making minicourses, complete with all 
demonstration tdble and booth tangibles, available in some of the mB\or residence halls. The Audio-Tutorial sys- 
tem n^»ed not be bounded by the institutional environment and indeed is not in our program. Our materials are 
already affecting those outside the institutional and course setting without out intentional design. Students involved 
with take home packages not come to us and say "my husband, brother, wife, njother or whoever listened to mini- 
course Opi 1 and wanted me to ask/' anrJ so on. Beautiful? Think of the effect we could have if we designed this 
kmd of non-institutional encounter. 
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Let us ask another question tho urgtifu/or?; ut this synipostum rcquusted we address ourselves to. 

What about costs of a tcchnolof|icjiiv oriented pro(jram like Audio^Tutorial, are they high? The answer is, 
they need not he — I've alreddy jHutiini to th<' low tcchm>lot|y »iSfJiH:t of A-T, In fact, one can move into A-T and 
stay almost entirely out of the hardware husiness. That sounds contradictory pL'rhaps, but students can prox'de 
their own tape players or buy or rent them from book stores. A reliable inexpensive playback-only device w^h 
no record furKtion can be made available to stucJents for about S20, With a two to four student cooperative 
venture this cost figure is not overly burdensome- A mjster recorder and dupficdting system, and the needed pro- 
jection equipment are the only pieci?5 of hardware absolut«ily tiecessary for an Audio-Tutorial program and many 
of these are already available on most campuses. 

On the other hand, one can go *'f irst cabin" and for 20 to $25,000 completely equip a facility with 32 
booth- enough equipment and space to tutor at)\)ut 5 to 600 students through a 4 semester credit hour course by 
remaining open 70 hours per week, hi the cost category, Audio-Tutorial is just not in the game with video formats 
for CAI- Perhaps to some potential innovators, th*? iack-lustre hardware needs are a negative factor of A-T. Hope- 
fully this is not the situation. 

Well t have had your attention fur u cunsiiierabte s|.>an of time and you have been patient. Let us end the 
A-T story here, overamplif ied in some areas perhaps anil underamplif ied m other where you may have some 
questions. I thank you for your time, but before I answer any questions, let me make my |X)sition clear. 
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Like my amphibian friend here, I feel fairly comfortable with my stance on innovation; I think I'm on top of 
things, resting on a fairly firm foundation and {M?rhaps sittinq a cut ^jl)OVf? those who, in spite of the virtue of their 
advanced design like the turtle here, still have to go ^ome to beat my position. IMow you ask, "what happens when 
this advanced design really moves?" I jump and if I |ump high enough and this turtle doesn't move too fast, I may 
maintain my position. Thank you, 57 



LOW COST EDUCATIONAL TECHNOLOGY 



David C. Miller 
Director, Instructional Resource Center 
University of Illinois, Chicago Circle 

INTRODUCTION 

A major concern regarcJit.g the future of educational techriology in higher eiiucation is the apparent low fevel 
of use parttcularly m the traciitional university programs. Because substantial funds have been devoted to encourage 
the adopf on of var.ous technological toots, over the past 20 years, most support agencies have been rewevv^ng their 
^ograms to see .f better ideas and approaches might be available. The recent report' prepared by the NAE for the 
Of ..ce of Education is an example of the studies that are under way. 

I used the term "apparent low level of use" to highlit ht the difficulty that anyone experiences m trying to 
determ.ne what actually .s the level of use and perhaps, as equally important. «^at is the rate of increased use 
Socal changes have time constants of decades and some sponsors of programs have failed to realize that short pro- 
grams, even those supported at a h.gh level of funding, may be counterproductive in the short term. Norman 
MacKm^.e has defined educat.onal technology as "the systematic study of the means whereby educational ends 
are achieved. If we were able to measure the rate of increased use of educational technology so defined ' believe 
this measure would provide an optimistic index of the future for technol<^y in higher education However the 
fact rema.ns that to date, the vast majority of students in higher education are being taught by a lecturer unaware of 
or uninterested in instructional technology who uses books and a blackboard as his only teaching aids There are 
important pressures being applied to change the situation because of perceived potential economies by administrators 
and legis ators. demands for better instruction by students and the desire of individual instructors to cope more 
ettectively with an increasingly complex task. 

What then are the constraints that slow up the more rapid adoption of technology in education? Perhaps a 
review of one or two highly successful technological invasions of education will provide some guidance The paper- 
back hook has become so common it may not be recognized as the great revolution in book publishing that it has 
been. While books have been a vital part of the instructional process for several centuries, the availability of low 
cost editions m a very convenient format for carrying to and from classes has made it possible for instructors to 
require studc.us to buy more than just one textbook in a course and thereby provide the student with broad readily 
available study resources. The educational need and familiarity with the product combined with low cost borne by 
the student, the easy availahility and convenience of use all contributed to rapid adoption and very widespread use. 

A more recent technological invasion of the academic world is the dry copy duplicating machine. The use of 
these machines has been so great as to give the publication industry great concern and has further complicated the 
already almost impossibly complex problem of devising a new federal copyright law. Evidently an educational ncerl 

rV/a'"""' ^""''"""'"^ ""^ l^"-°""V 196fi 1070 Cen.r. lor F.U,c.,t.on«I Torlmolof,^, Un.v. of S.,«.«. Hr..,Mon, 
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exists fdr the rap*d copying of many dcMiument?. The dry copy process is dean and quick and except for photo 
graphs gives excellent copies, tt is simple to use and available for use by any instructor at his conwEnience. Finally, 
it usually is used with an illusion of no cost because the operating cost \t covered by an overbed account and no 
m jney passes at the time the oopfes are made. Furthermote, the per copy a)st actually is relativeJy fow when only 
small nuniuers of copies of any ^ven docufm»nt are needed. 

The conclusion we can draw from these two examples is that tedinologi^al developments can penetrate 
educational institutions ami do so fairly rapidly if they satisfy three criteria. F they must satisfy an educational 
need as perceived by instructors. Secondly, they must be readily available and convenient to me. Thirdly, they muvt 
be relatively low in cost to the student, to the instruaor and to the institution. 

Low Cost Technology 

With these thi tn? criteria in mind, what can be said of the technological systems that educators hav^ betii 
exposed to for the f)ast twenty or thirty years. Few, if any, have met all three of these goals. The cost criteria may 
be satisfied if the cost i»er student hour of system use is low, say 50 cents or less. In that case^ several of tfw 
computer hasfxi systi?int, may operate at this cost level. However, capital intensive systems in tl^ academic en"<ror< 
ment have difficulty \n "aising first costs, creating and maintaining the operating and repair staffs required and 
perhaps most serious, do not have approfKiate aoKirtization progrants to cope with rapid obsofescenoe of complex 
equipment. Even if all of these capital and operating budget needs are included in the student hour cost calcula- 
tions, unless university accounting practices recognize the sp^^cial nature of these systems, the necessary funds for 
replacement and maintenance are not made available and long term operation hccomes irKreasingly difficult. 

Obviously, all systems involving software and hardware require capital investn^nts and operating support. 
However, there is a spectrum of costs of this nature and in tho^ programs that hat^ been showing steady grov^h 
over the past ten years without major forced feeding from the foundations, the capital investments are relativel , 
modest. One of the best examples is the auto-tutorial system whidi is being discussed in another paper at this 
symposium. For some time now, I have been studying educational systen^ that incorporate low cost techrfology 
to see how they could also meet the criteria of perceived need, availability and convenience. Two modeis, each 
of which is adaptable to conventional institutions will be described. All of the low cost technological innovations 
of which I am aware can t>e used in one or both of these models. 



LARGE CLASS MODEL 

Each model offers a way to increase the student faculty ratio. The first model uses the obvious approach and 
merely increases class sizes. This is not a new approach but concern for the quality of education in large group 
lectures is common among educators and students. There are lecturers who cope successfully with classes of several 
hundred students, but many instructors are not comfortable with such large groups or are not particularly concerned 
with the effectiveness of instruction in such circumstances. Sint^ such classes are the delight of admipi«;tratOfs arid 
deans because of the massive FTE contributions they produce, the use of large dasses will fM-obably i-ontinue to 
increase. However, as higher education continues to be faced with static or even shrinking enrollments, student 
concerns about instructional quality may become increasingly important and if so, these concerns wtll dash with 
administrative desires to use more l^rge classes. 

A possible resolution of this conflict of interests could be achieved by the more extensive i»e of technology m 
these large clasb situations. Again, thi^ is not a new idea, it is being done and has been done for many years. Ho^/evrr , 
the extent and nature of use varies greatly. I am sure my premise that technology can imf^ove effectiveness will be 
challenged and I realize that the research literature on this point is inconclusive However, the rapid move toward 
highly flexible curricula provide students with the opportunity to express their preferences in the one way that is 
uncontestable; they can and are refusing to sign up for poorly taught classes. This ir. still a new pt.enomena and the 
effects arr mutfd t)y thf} still lar<|f^ rpsidual of ''refjuired courses" hut tN? trend toward a more open rurncuhim is 
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acceterdlfng and [XotectiHd havem are likely to ctecre^ Jn nunrt^r quite rafMcay . TT^refore, I am ca^nfkhfnt that 
courses using technology ixoperty wM continue to attract students and thrs atone wiU wrif y its wintribution 
to tfVe ''effectiveness'' oMt^ instruction. 

Utt of Mot KMI PictufM 



The term ''usUig tedimjfcw properly/' is a key concept. In terms of Xtnf twvioielv noted tbree criteria-cost, 
convenience and perceived TOed--what technology is best adapts to tarcp group instruction? Starting from the 
consttleratJon of perceive ne^, the tmtructor should desire to ^in amt hold attenticm. The most wkiely used 
technology with the proven caj^ility of holding thf attention of wry *arge audiences for sewjrat hours at a tinw 
is it^ motion picture shown on a big screen. Holdir^ at^ntion, however, is cmly a start on the learning process 
The dvail Aility of a iar^^ cotor iiK>ving ima^p that all students can easily see can also satisfy a nuit^r of other 
instructor needs by |w^o\Hdii^ unique learning experiences. Sk>me of tt^ mone* c^ioi^ exarrq^es c^talize on 
bringing remote or uni^ual or dangerpi^ environnwnts Into ttw dassroom. of the rm)st successful til ms^ 
usetf on the UlCC campus was by fhrof- R. Simpson of tN* Potiticat Sa'ence Etept. arKi chronicles in some 
detail the organization and conduct of a pditical Cimpati^ for a state level office. 

Another important caoafaility of film is to allow a dass lo retiift; a ttnrm et^nt in history, fitter from mi 
actual filming of tlw episode or through a dramatic recreation. Technic^ uses tndude a wide variety of demon- 
sf ration experinwnts that ffin^irv no set-up time arid always \wrk. In this re^^d. the ability of the carrtera to 
tnagnify or see into normally inacarssible environments can also add sulistantially to a lecture fKesentaticK), 

Most people are aware of the edt«:ationat possibilities ot film. Why then is it not usednrwrc often? f^rhat^ 
a review of how the me of filfVi fail*; with regard to conwirience of use, availability and cost vyitl help to darify the 
Situation. If we linvit the us*; of f*lm to large lecture dass^, showing of the film may be reioonabty convenient. 
Many large dassrcK>ms havtr i$ iav^ screen, prjectot, and e^, in some cases, a fH^ojectton booth. A projectionist is 
v<*ry helpful if the film is used fm only parts of the di^s p*?rf ud and niay or may not be available, Finding and 
oinatning appropriate ftlrro can be a s^iom prol>lemt but if tho camiius h^s a good reference service, data on a 
Vkride variety of available films can be provided. However, if thii film must rented from off<:^mpm, scheduling 
becomes a factor end decision toad tirm may be vie^ft^ by the instructor as a serious incon^ittence. If preview- 
ing of films is necessary « additional time is niquir^ and the ixin^niencs factor suffers still more. The wst of 
a rental film averages about SI 5 a showmg wd the a>$t of providing an operatic may another S& to S10 to 
this amount. If ten films ai^ ust n a given course ^urin^ a semester, the total cost vi^tild be between $200 and 
$250 or, for a lar^ dass. about SI per student fmt course. 

If custom film making ^required, the cat increases siibstanti^ty. Commerdat film f^ioduction ctKts about 
S2,{KM} per minute of finished fHm and evwn low cost campus production if fully costed can run $400 per minute. 
However, an instructor may feel the need for a specific film that can sen® to tie together the other films used in 
the course and a major f»rt of the lecture materials. Assurrsng h film would t>e current in content for 5 years, a 
30 minute film locally produced woaid add about S2,400 to the course cost or about $10 per sttKlent f^r course. 

To summarize, if tfw nenfd exists to use film, the convenience of use and avails^ility is only fair, while tim 
cost ran^ from low to fairly high. Gi\^ optimum OMiiiitions film can and is f»'oviding strong support for 
large d^^s. Unfortunately optimum conditions an? not widespread in hi^r educational imtitutions and the 
cost, inconvenience and unavaHabiJity factors militate against such use ' 

Slides 

An alternative to the use of film is the 35mm slide or ono of itf, variants. The list of possible jwrceived needs 
for the use of slides is similar to the OfM? <tetailed for film. The lack of nwtion does introduce an important 

•^Sv ttifl P«0ple. prodiicod by H. Simpson, d^r^ct^i hy W. Dtstfiimted by Pjtm Inages^Radim FHfns, 

R«lflasotl 1972. 
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hfTHtation. but can Im? partialiy compensated f<^ by the i»e of a spect^if techmque^ that tvw projectors and a 
lap dissolve svvitchiny unit. By using alternate superimposed images f rtwu two sihte fwcqectofs and secH»?ndng tl^m 
rapidly, an Hfusioti of motion can be gm^rated that will suffice for nf««My af^tcations. &mn for just sho^ring a 
sequence of slides ai d slow para, ttie te<^qi^ dtlo%vs the same sort of ti ansition from one Smag^ to the next that 
people are au:ustOfiH;<l to tfi fUn« and tefevm^ ri 

CN>ce more hov^wr, J must ^mii that slides are not used extensively in hi^r education ami while there 
does seem to be a trend toward ino^ased us^^ a tmtm pr(4iiem is stilt inconvenience. Actu^ vtewing of slictes in a 
big classroom may rK»t be mudi of a pri^em and b dissol^ unit may even 1^ availaMe* The prc^jectcm c«2n be 
renK>teiy contrctied by tl^ lecturer and trays of slides easify loaded and unloaded, ffovw\«r« it is in the prepara- 
tion Of procurocnent of sllctes that the major inconvenient ensi^. Side collections can be purcha^ but content 
and avaitaditity usualty more difficult to determine than in the case of film. If purdiased^ slictos wilt ccsi about SO 
cents to $1 each and if they are issed extensively in a course, this can exceed the (x>st of a film rental. On the 
other hand, slides can be used many tintes and collections of slides can be easily edited, so the long term cost is 
probity lowt»r than for film. 

(f tt^ instructor wants to use custom-made sitcfes. he must f»^^ovide tf^ raw data. TImn tte slides will cost about 
$2 to $3 ea(^. This cx>st can be reduced by the do-it-yourself instructor, but ready access to art materials, drafting 
t^les, varttypers and copy cameras may a i^tAlem ar^ in any case wilt comtitute an incxinventenoe« Cor^idsring 
only film and processing c^ts, a slide mt^t f^epared for i|s little as 10 cents. If « hcwK^, tha lap dissolve 
tec^niq^e is used« two SO slide trays qan be used during a sins^e lecture and would represmt a first oast of ^ut 
Assuming a once a year use and a five year fife* the maximum cost for forty two of^ hour dass sessions would be 
$1 35 or, again, foi a hr^ class ali^ui $1 per sttKient per course, tn most cases^ sticies would cost imire than this 
n^imum amount but rntensity of vMHikt prdbabty also mudt tower« so tfiese tv^ facers would tend to cancel 
and the $1 cost fiqure is probably a good average. 

To summarize for slides, they can meet many of the same needs as film does. They are more convenient to 
show, more difficult to buy or rent and easier to custom design and make. However, slide makif^ fs a time consuming 
inconvenience great enouj^ to discourage most instructors. Costs are about the san^ as film rentals and could be 
considered tow if used with lari^ classes. 

Overbad Pru^ectors 

Probity the most widely used classroom pfesentation de^ce is the overbed f^ojector. interestin^y* it also 
confines the greatest convenienoe of with iow^t operating cost of any largp i^escmtation device. 
The fxrojiector is totally iHtder a>nrxH of the lecturer atui can be umi as he wotM im a trfackboatd* Traf^^iafiencies 
can be readily prepare prior to da^ on fdastic she^te that cost about ten cents ff er^^irfe markers are 
u^, the sheets can be remed many times. Colored line drav^n^ are e^ to prei^re with appropriate per». 
Colored pictures are fx^ible but ref»«ent a mi^ irare difficult f»rcrf>tem« Movirq inr^^ ^ not pmsible in the 
film mtKte but motion of the pen in drawing curves or v^riting usually imf^rte a hi^r intrer^t level thw a still 
imag^. Students often object to the t^e of prepared transparencies on the overfte^f projector because it allows 
the lecturer to present materials much fmtBr than they can put it in their notes. This f»t>blem can be overtxtnw 
if the lectufer also provides copies of notes covering everything presented on the screen, but this involves extra 
time and eliminates what n^y be an important learning pN'ocess. An Alternative solution is to p^ ti^ presenta* 
tion of material to allow time for note taking. 

Rated in order of low cost, convenient and availability, the overf^^ prelector is dearly die first choi<»« 
the aiide (^esentation is secofKi, srui nK)vies con^ off third, f-towever, a \^!-t^tOTC^ presentation strategy 
couid find unique need for eadi of tl^se nwtes in any giwn oour^. The ability of the film to engender affective 



^Xap Oifiso^ve Pfo^tion; A Tecnnlque for Teaching Spatlel. Temporal and Orgsnlfstloit^ R«liitHmsMps/' J, S. Daniel, 

J. P, Ballon, U. of Montreal, Proc. of Third Annual Frontiers m Education Conferanca, n£££|. p. 297, 
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as well as cognitive responses is particularly important and often is not fully appreciated. Properly orchestratCKi 
these three presentation tediniques can be used at a cost level that is acceptable and, by so doing, can ^nerate 
a high tntcrest level and t>roaden access to content thereby making the lar^ class format quite acojptable and 
eilucationaily profitable to stuitents. 

The use of television in the large class situation has been purposely neglected. Projection television is 
exfxmsive, technically complex and untif very recently, has lacked color capability. Use of individual monitors 
scattered throughout a targe classroom can provide an alternative presentation mode, but I object to the small 
screen and often poor color that such television offers. Future developments may diange the situation, but in 
that case cost, convenience and availability will be very similar to the film situation. The only fx>ssible exception 
might be when a closed circuit tdevision system is used as an overhead projector with a camera focussed on the 
desk pad of the instructor- While cheaper than filming such an event, the cost is still not competitive with a simple 
optical overhead projector. 

INDiViDUAL OR SMALL GROUP STUDY MODEL 

A seconii model of an instructional system that can use low cost technology is based on the assumption that 
the major fraction of student learning occurs in the self-study or peer group mode and that if efforts are expended 
to structure and aid this It activity more effectively than has been the case in the past, an important amount 
of time that has been devotev. information processing in lectures can be eliminated. The auto tutorial system, 
which is a specific execution of this model, has g\ven the impetus to a wide variety of related approaches. A 
common pattern in all of these systems is a reduction in formal lecture time and varying degrees of individual 
student assistance. 

As noted in the large class model discussion, the only way reasonable costs of technical sub-systems could be 
achieved was by spreading the costs over a large student population for each application. Ewn under these circum- 
stances, some costs such as those associated with the production of custom made films are difficult to cover. There- 
fore, if a system of instruction proposes to provide technology mediated learning for individuals, the necessary 
duplication of both software and hardware becomes a very serious problem. One of the first systems devised to 
provide individual instruction was the language laboratory. Unfortunately, the inflexibilities in use that were intro- 
duced by cost considerations have contributed to its reputation as one of the less successful educational innovations, 

So the challenge of designing successful systems that provide individualized opportunities for guided self ■ 
study depends heavily on low cost people and when appropriate and available, low cost technology. The use of 
differentiated staffing, even to the extent of unpaid undergraduate tutors has provided a component of lower paid 
instructional staff. The problem of providing lower cost technology has been more difficult because most of the 
har(iware was designetl for large group use and even software costs were justified on the basis of only one copy 
being needed for a class of thirty or more students. 

In regard to projected Images, the ultimate low cosr device is not yet available, although there are some 
systems such as the Philips PIPS units that recognise the need and have made a start in the right direction. 

Audio Tape Cassettes 

In the audio field, the cassette format has made possible a variety of applications that support indiviciual 
stu(iy efforts at a low cost. Several applications have been mentioned in previous papers^^ but for the sake of 
completeness, will be reviewi?d. 

^"A SystsfTii Approach to indivtcluah^uti Instruction." D. C Miller, ProcQeilmgs Second Annurl frontiers in LdocoUon 
Conferonttt, 1972 (lEEEl 

^"Tochnologv and Self Sttifly," D. C, Mdler, J, Educational Tecfmology Systems, Vol. 1 til pp. 73 80. Jun*?, 1972. 
^"ThM Aufi<o Tapo CaSSeTTr* fn tducat.un," D C. Miller, f ngmoiinng Ldutation, Vol. G3, No, 0. M^fch, 1973, 
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Perhaps the technical sub-syslefTs having the lowest costs to the institution, are thuse tfiat can use student 
suppiied equipment. Audio tape cassette record/playback units are widely available and less costly than a tyj^- 
writer. If the assumption is m^e that eawii student can find access to a playback unit, cassettes can used in 
several ways to promote the eHectiveross of the self-study mode. For example, immediate ansv^rs to problem 
sets that are assigned in courses in math, science and engineering can be provided if a commentary ta^ and an 
answer sheet is given to the student when he harnts in his work. This enables the students to immediately check 
their results, alternate methods of solution and relationships to lecture materials. The instructor then need only 
dieck who handed in a paper and spot check some of them for feedback. If graded papers for all students are 
considered essential, the students can be asked to keep a copy of their solutions, ^-ade it and report the result. 
Again, a spot check of the papers turned in could keep the resfwnses honest. 

The trade-off, of course, is the effort required to prepare the tape. Howe^r, once a master tape is pre- 
pared, it can be reused each time the course is taught. An average cassette commentary is about 30 minutes, so 
the production time should not be rm^re than an hour. There is r^ need for scripting although a few notes on each 
problem is helpful, p^icularly if previous feedback from students had indicated difficult com^pts or computational 
"booby traps." Often a tape dialogue is a good format, where a student or another instructor serve as a surrogate 
for the listener by introducing questions or comrr^nts. 

if the campus has its own audio studios, the recording can be done there, if not, tapes can be made in a 
classroom or in an office if the instructor has a rea>rder. The (Kincipal cost to the school for sudi a practice is 
the duplication of the cassettes. Cassette duplicators may cost as much as tfiree or four ^ousand dollars, but 
can be used with student labor. Thirty minute c^settes for a ttdSs of 40 student could be duplicated in about one 
hour depending on the particular type of duplicator used. Cassettes can be provided by tt>e school with an 
understanding that the student return a c^sette for each different onn he t rceives. Thus, the school need only 
buy as many cassettes as stcKients in the class. If a student wants to keep f^ne or more ta^^s, he is asked to sub- 
stitute a blank cassette for each one he wants to save. Tapws for person class can be purchased for about 
$35 and can be used for four or five courses in sequence. The cost per student hour for such a system would be 
5^, so it is truly low cost. 

Learning Laboratories 

If the school is able to set up learning laboratories where students can come to listen to prepared tapes, 
using them in concert with visual materials, the capital costs are somewhat greater but still low* A thirty carrel 
unit could be assembled for $20,000 to $25,000. Assuming 12 hour a day availability, the cost per student hour 
could be as low as seven cents although 1tX)% utilization would be required to reach this figure. But even at K)% 
utilization, the cost is low. However, the cost of software must be aided to this capital wst. If the school 
provides the learning software in the tape/slide format and each set requires ^ut one student hour to complete, 
ten copies of each set could serve the needs of a class of two hundred. A tape/slide combination could cost about 
$30 a copy and could be used for at least five course repetitions. So the cost per studem hour use is about $.03. 
Costs for preparing the software should be added to this figure, if the materials are not (^nrimerciaily available. Using 
a preparation cost of $500 per tape/slide set, and again spreading this cost over a five year period, the cost per 
student hour of use would be about $.50. Added to the capital costs and duplicating charges, this cost level would 
be marginal in meetirq the low cost criterion. The software preparation time is the major cost fsctor and is seldom 
fully costed, since the instructor will contribute time and often campus media services are availablt** from overhead 
accounts. Thus the perceived cost usually does fall within the low cost category. 

It is also interesting to note that for a given tape/slide combination, the cost of the slides comprises about 80% 
of the total. The 35mm slide is expensive to use for individual viewing and alternate formats are needed. The film 
strip is one such alternative but iack of highly reliable display equipment, fjlfficulties in editing and the high cost of 
small numbers of copies are serious limitations. 
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Some applications in th^ te^nif^ iatoratories do not i^«d or lee {Projected vf$u^ K)ftw^ ^ depend on 
audio cassette* ami accompany ii^ printed materiah. SiK*i a a>mbjnatton not on»v fireatty reduces cost but scmw 
apiirfications can tie the best mcwle of Irtttruction. **Hm^% on** experience is often the best way to hwn how to 
operate a wide rang^ of de\n<^ and the t^/worktKic^ is p^tciit^y vimlt ^apted to ^ist siH^ te^ning activities. 
For example, familfi^zation with etectrcwiic t^x^gtc^ equtptrant such ^ osei{loscope or a si^ i^rator h 
wry effectlwf y awiducted by use of a cas$ette/worfctK>ok a>n*ination ami sever^ venuor^ have teen pep^^ and 
used. Some video enthusi^ts ^ that tf audio ci^ttes me ^wf^ vkteo cs^tt^ mwt t3»tt^. I think that when 
a fewiing situation already require full visual attention by the itiKlent. to offw wn^ting visud information may 
stow down rather than aid the learning fmicess. It ^rtainly adds to the TOSt. 

Int«raetii» Lettutes 

of the eari test i»es of the audio a^tte reocNtfers was die reocmiing of ct^sroom lectures* Most students 
and many instructors learned that diis procedure was not the learning boon It w^ expect^ to be, itetmit^ to 
^io can be very dull ami without a visually related im^ t» siAject to many cKstr^kMis the visu^ 
demonstrates its domin^ice. Also a or» hour tecture r^)uln» one hour to re^ew. Thus, the abHity to f^ovtde 
individual oral versiom of a lecture at a lew €xmt was m attri^l^ scrfutfon in search of a problem. One of the most 
inii^restlng ^proaches to resolving diis situattcm has been d^ ^'interactive l^we*'^ developed by tk. Stew»t 
Wilson of die Polarok) CkNfporation. 

Most mediated ins^uction gutcfes a student throi^ a lorqf series of ieptiir^ s^f^ depends upon the 
freqt^nt iro of program j^^edfied cpiestions to fmvide tear^if^ leenforcement* Hovtfsi^« in nwiy sucmsfut learn- 
ing environ n^nts, the inverse occurs-the qimtiom ^estu<fent fc^mufated bai»d on his rate of conq]^i^n$iOT and 
(Hievious faa<*^ound. The system ctevi^ by Or. Wilson oeatesi® nearly ^ possible with media; a situation v^reby 
a student can learn c€^tent tnf Hstenrng to a lecture, <»i initiate ^H^^roprtate questions of his dKK>sing and readily 
obtain answers to these qtmstlom. "Hie system is based on the concept diat for a » ven lecture^ there is a finite num- 
ber of questions that any student hearing the tecture is likely to ask, 8y having a reason^ sized sam^ite of 
students listen to 9 given lecture ard list dielr questions, a b^nk of questions cw be ^nerated. Furthermore, it 
seems reasonat^e to expect that if a tar^p number of students participate in the prepiaratimi of the question bank, 
any qua^ion 3 stident mi^t have sidi>5^^ntty while listening to tl^ lecture vraufd hmw a hi^i probability of 
being in the question bwk- Then, by renfrding on cas^ietfes aitswers to ait the qimtions in the bank and arranging a 
method of fairly rapid access for any s^aedfic amwer, tiw system can provkte the student with a low cost Interactive 
learning resource. 

Dr. Wilson also uses a modi^ Flexiwrit^ whicii operates from (Kie track of die c^sette and draws on a 
sketch pmi any visual materials the torturer wish^ to use with his pn^mitation. 

i have adapted this kiea to a ^>nmi^at 1^ sof^bticaled system with d^ ho^^ of nudcing it more at^active 
cost^wise ami sonw«i^at e^ter for a stt^font to u^. By reccing instruction^ inures* editing out all questtons 
^ked by sttKlents and repetitive or irrelevant materials, a fifty minute l^ure can usually be r^c^ to about 
thirty minutes. Using speech comf^e^ion equijimient, tte lecture can be ftrther reduced to atx)Ut 25 minutes. 
A vwrkbook is (spared for the vfsuri materials diat ac^>m^ny die iMture. Iteing the question set generated by 
students during the lecture ^ a nudei^, additional stt^nts are asked to li^en to the coiKtensed lecture and mid 
questions. The final set of questions and answers to the^ questions are put toother as a pat of tb^ final workbook. 
If desired the question set can be added to and answers can be modified ^ the lecture is mflrfe avaif^e to aciiiitional 
students. Thus the unit can be kept up-to-date with relatiwly little ^itional effort. / 

The interactive lecture in thfe nwdified format is partlcuJariy suit^ for individi^iring imtru^ion. The only ' 
ccMttributton required of the if^nictor is to give permission to record his lectures and to write ^iswers to tl^ 
questions In the question bank. Hov^i^r, the tape editing must be done by sonmme quite familiar with th#» 
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lectun? content and consultation mth the instructor is essential. The system is particularly suited to optimize 
student time since a bright student can spend about halt as much time listening to lectures hf nor mally wouM 
whereas the slower student could review all of the available questions if desired and never lisk exposing to the 
instructor his inability to quickly grasp ttie material. The portability of the tape/workbook Cvimbination allows tho 
material to be studied almost anywhere it is convenient for the student to do so. ^/^^ 

Unfortunately, while the system is low cost and can be used conveniently, the perceived need by instructors 
has not yet materialized in any significant number of cases. One problem is the acceptance of the desirability to 
rno\^ the data transmitting function from the group lecture mode to the individual self study format. Another 
problem is the availability of skilled tape editors who are also rei^onably krK)wtedgeable in thf» content to 
edited. However, students can learn editing quickly and can do a fine job on content with a little experience, but 
convincing instructors to use the technique is much more difficult. 

Calcuiatois 

No discussion of low cost technology can ignore the potential and real impact of the new solid state calcula- 
tors, particularly the hand held units. Costs range from S50 to $400 depending upon the sophistication of the 
calculations desired. It seems likely that almost every student that takes courses involving calculations sooner or 
later wit! obtain such a unit for his personal use just as studen**. now buy typewriters, slide rules and cassette 
recoi ders. 

The impact of the greatly increased computational capability these devices can provide to students will 
certainly allow a substantial change in the lewl and nunrtber of problenr>s assigned in tedinical courses. In those 
schools, such as Dartmouth, where students have had ready access to the computational power of the digital 
computer, substantial course modification has already occured and might serve to guide future steps in this 
direction. 

Cost and convenience would lead one to expect a rapid incorporation of these inexpensive calculators into the 
curriculum but, again, perceived need by instructors may not be sufficient to push the acceptance of this innovation 
as quickly as some of us would like. 

Television Monitors 

The uje of TV cassette players and color monitors is a very attractive presentation mode and is receiving a 
great deal of attention at the present time. I think it is particularly well adapted to use by individuals or perhaps 
small groups. However, the cost of a playback unit with color monitor is at least $1,5(K) and each unrecorded 
cassette costs about $25, Therefore, 'hile I hope money can be found to outfit at least portions of learning labs 
with TV cassette.' players, I cannot classify it as a low cost technology - at least not in the near future. 

CONCLUSION 

Two models of an educational system have been reviewed for needs that could be satisfied by some form of 
technology. Some needs were identified and possible technological solutions were discussed. The criteria of low cost, 
convenience of use, ease of availability and perceived need by instructors were applied to each technological 
approach and the degree to which the technology under consideration met these criteria was assessed. None of the 
technological systems presently available met all of these criteria to a high degree, although the overhead projector 
use m the large group model and the audio cassette use in the individual study model came close to satisfying the 
criteria posed as necessary for successful widespread adoption. 
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f n ipost c^es, measurement of the effects ot new <h^ expenrwntal imiructicMiat technology involves the 
assei^mtiffU of changes in the subjects undergoing the s(^iai tneiatment. &fdcKn is them an exannination of ottiers 
in the ettucatfcmat envtrwin^t wttti ^cmr either the sublets or the tedinologyttsetf interf^. This paf^r locrics 
at some of the $>otemfal feiattonships between the subjects in my experimentaf instri^rtionat ^ing and stiKfents^ 
facuHy and admimstratoni whc^ relationship^ to tfie new technolc^ are initi^ly indirect mi tan^fntial. The 
paper is concerned iwticuiarty widt diese letationships in a ctmventtatiy boum^ hi^er educattcm system whether 
i^fw teaching^teaming technotc^es represent onty a refativety smati input to the total environment, ft concentrates 
mostly on tf>e imp%t on st^Kfents. 

The paper be^ns with a discu^ion of some of the variables which mig^t tie expIcK^. fn particutar^ raeds of 
students and ^itm of the curriculum in a c<mventional sysusm are conlrasited. Next, the typicaf cAqectives of new 
instructional technologies on these campuses are discussed. An attempt is made to shc;v how these t^nologpes 
usually assunr^ a limited set of aims at tr^itit^d cdleges. The nature of mvironment M^tdh is neected to 
support the broader aims of liberal ediKzation are dien su^ested, foilov^ uy some hypotl'mes ^imit die imf^ 
of new tnstructicmal technologies w that envirwrnent. A short section on environmaotaf tne^urwnent i* dien 
offered. The paper conclu<fes with several recommendations about how ted>nolo^es must be adapted to recognize 
not only curricular aims and student needs but also the environmental ccndttitHis vA^kh are condi^ive to dieir 
satisfaction. 

Gftmyng Neeife of SUHtents 

One way of evaluating the effect of new instructional technologies on a stuttent body is to determine if and 
how they meet toth the aims of die i nstitutic^ of hi^w eckicattc^ and of die neecte of its students. Oirricula in 
post-secondary education are typically desired v^th each of two fvoad sets of objectives, ttHiu^ it is sc^netimes 
claimed they are in conflict with one mother. Depending in f^rt on the kind of instituticm, ocmventional courses 
Pressed at meeting ir^tltutional needs are intended to h^ an innmediate e^ct on the student's store of 
knowledge in a particular field, which in turn, it is presun^, allows him to be more adec^ia^ly prepared fm his 
post-f^aduate life. In ackiition, conventional courses have the objective of incre^ing the stuctent's critical c^sacities - 
his abilities to think objectively »6 creati^iy. Freqi^tly the curriculum as a i^ole attempts to instill stmte 
awarene^ of alternative value systenr^, particularly ^ these relate to die duties of ^>od citizen^ip. It is hof^ that 
die stiKitem will, in addition, benefit from his courses by having his o^rall awarer^ ami appreciation of the firo 
arts enhanced. Finaily, c<»iventii^al curricula often are desired to allow students opportunities to It am to act 
with independence and autonomy. It should be noted that these instituticKial obiectives look mosdy to L^'• prep^M^- 
ation of the student for the future, radier than to his de^lopment in the present. Moreover, die pedagc^cal style 
which has appeared most cc»iducive to learning under this assumptic^ is one in lA^ich learning takes place through 
the ^quisition of "bits'' of information, arranged sequentially according to any of a number of theories of 
co^itive development. 

But students often have their ovw sets of needs in addition to diose con>fT>onl^r <ts<wmrt by institutional ec^ca- 
tional planners in their desi^ of formal curricula {Kau, 1968; Chickt^ring, 1969) Younqf^ople coming to college 
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are seeking to liml out who they are and what they ^ ''foorf' at. More than simpiv wcationai ictentification, 
college stuctents took for a iSmpm sense of the continuing st^e »t qf personality traits they can caH tfietr own. 
They want also to explore aUemative Mving styles, often a frig^itenmg iHOSpect for them in ytftiidi internal con 
flicts lead to confusing tergiversation. Students are interested in de^toping their interpersonal oimpetencies - 
i,e., their capacities to communicate intimately with ott^r human beings. Similarly, they are confirmed with the 
ways they handle ttieir emoticwis, desiring to fed more ccwifortaWe w^en tiiey expre^ themselves o{3«nty or when 
they find it necessary to be restrained. Fir^t4y. ^y search for mi^ningin their Hi^. A neverending «fe4<mg 
(^tjn the college years, tfiis drive is particularly salient. 

These two sets of objectim {of the instituticm and of the student) are de«^ly related, but often in ccmtempo- 
rary hi^r education they are unfortunately opp<^ to or« mother. The mofe subtle and often umtefined and 
usually unstated educatic»ial cA^ives of tfie institution are translated into a formal curriculum^ ich is so geared 
to the meeting of students' future needs that it may Ispior^ dic»e miKt urgently felt fay the stuctents as ttiey fntrsue 
their daily lives. That ttiis ^parmit dualism is nrore myth tfian reality was, of ixHJrse, Dewey's ihen^ in mw*t of 
his writing (e.g., I^wey, 19\B). The artificidi sef^ration of future mtd pr^ent of institutitxi wut student, of obfect 
and subiect is ^entuated by tfie differences am<mg the disciptin«& where f^lty have quite different p«dagc^cat 
assumptions and obiectives (Gamsw, 1^7), The dichotomizatiOT of ir^itutional and student nee* teaV^ un- 
examined the relationship between nrotivaticm and learning, beti^een emoticm and re^on ^d between the urgency 
of the student's present and tfie pr^icalities of his future and *at of his socfety. One importi^t test of new 
educattwal technologies, then, is to see how welt they address them^ Ives tothi»« interfockingsetsof needs^ To 
the extent that they can n^t this challenge, they wilt hwe f^litated tfte needs of the society for m educated $nd 
informed citizenry, the needs of the students for i:^rsonat yowth and cfevei<viment and the needs of a |Xid^|09ca< 
system which accounts for the relationship between the two. 

It seems reasonably clear that particularly in bcHincted^ convention^ systems of hitter eckication, designers 
of new technologies have not ade<^tely c^{^ended die synerg^tie nature of institutionat and student needs and 
have chosen to concentrate on the former. Tfie impetus for experimentation in |]^dg<^cat forns on these campuses 
has been spurred by financial pressures on colleges and Universities to find ways to teaOt nmre ''efficiently'* - 
i*e., to reach out to greater numbers of students at less oist. It can be demonstrated (thouf^ it will not be here) 
that in back of these ttemands ft)r economy are the restricted set of ^surr^ticm bout the purpc^is of the curriculum 
noted in the first of the set of objectives c^ritmJ above {ue.. tho^ that are oriented toward tfie preparation of the 
student as a iuture cit ii^en). To the cfegree that teadrers and r^^i:hers are naively r^ponsive to federal agencies and 
other who would sp<msor such efforts, the experimental teciinologies will be geared f^crfominantly toward improving 
the more pastoral f urK:tions of hi^a^ education - the p^ing on of information u> students for their futun» use as 
adults and citizens. This cc^ttive and future^ooking orientation ne^ects not c«ty the Itnka^ between co^itive 
growth and total individual deveiof^ent but the manner in v^ich both take ^a<^ in a complicated s<K;ial context 
outside the ctassr^^m, 

SiipiKirtivG Educatimiai Envinmments "^^^ y 

i; 

A much more comprehensive ronceptualization of the mvir<^mental conditions requisite to the satisfaction of 
t)oth affective aitd co^itive teaming seen^ to be n^ded. Wt^ new instructional technotogies tend to reinforce the 
more traditional objective of curriculum makers, they assume a nwre limited role fcw^ the xsmA^r wd demand far 
less of the institutional culture in support of student growth and development. Computer assist^ instruction or 
self-paced learning devices tend to {thou^ they do not always) iipnore the deeper n^ik of Ihe student for personal 
growth. While they may recognize in fussing that Sfuttents may cte^lop ^eater independence or that feelings of 
personal a>mpetenc^ may be enh^ced when studmits proceed at rates compatible with ^ir learning abilities, such 
benefits are usually treated as fcKtuitous events rather than as achievemmts of piann^ c^iectives. 

Both apocrypha and research (Newcomb, 1962; IStewcomb and Wilsc^« 19^) sui^rt the view that peer 
influences as well as other non-curricutar experiences contribute as much to stuctent growth and clevelopment as 
does exposure to formal course work. For example, Bolton and Kammeyer (1967, p, 80) report that in bull ^sicms 
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out«(ie of dass. students tdk about inteHectual matttis onlv atKHJt 9% of the time, about fK^lar culture artd 
campus topics (indudfng academic mattere} about 24% of the time and^KXJt personal t^cems for the remaining 
two-thiith of the time. Despite Newcomb's strona ury ngs in 1962, littte research has been done (or at feast 
reported) whidt documents the kinds of effects su<rfi interacticHis ha»/e on1c«g-temv stuttent {^ovvth aid dewelop- 
ment. There is strcBig re««i to believe that in "bult se^icms" per se (i.e., not "study groups"Kthe intellectua! 
activity whtd) talces f^ace di»s less to aid students in acquiring the kncMfieti^ prescribed by the currtcufar ot^- 
tim thwi it stimulates, provikes, challenges and teadws values a«l beliefs c^sefating at a nwre profcnind lewl In 
the individual's motivational and p^sonality nexus. TTiat is, students are more likely to ei^age in convmatiwts 
in bull ^ssiwis v^ith xoudt on their deeply felt feelings about i<»t«» of importffice to them - feelir^^ whidi fre- 
cfiffintiy stem from prior socialization in eartier parental md education^ settings. Yet these intell^u^ exchan^ 
are of imiwnse value when they can be related to the airriculum, esfKcialty whai oirricular (or institutional) 
f^jectives are addressed to individual needs. As Newcomb (1962) iwtes, "Teachers' influence, if it is to be 
effective, must be caught up in the nonra of student s^tH^M. . /' The r«lati«^ip betwewi K^emic and non- 
academic life must. In otfier vrards, be amsdoiKly considered in the pteining of educsttonM ^j^!\ffiS. Needless 
to say, in tKunded. amventional systems of hi^er educatiwi, sudi connect|<»w are r««tv matte. The climate for 
establishing new instructional technologies on sui^ campuses with tiiis relationship in mind is not. therefore, a 
favoraUe one. 

Anotiier exvnple of stu<tent «a;ivlt¥ outside the d^room is in "dating." Thcnij^ on different campus«», 
interaction with the opposite sex may be called fay oihe. nwtjes. Bolton and Kammeyer at the University of 
Calif omia-Davis found that ttwir sample of sturtents aver^pd about 17 hours a ««ek in tfiis activity (either in 
dyads or in larger mixed-sex groi4>sK Their finding that students on dates tend more ihw in butt «sssions to talk 
abowt seritnis and self-revealing topks is not surfKising. They report: 

On dates, students are more likely to discus personality or identity pri^ems, long-run 
orientatiiHK, and intellectual topia, ei^^ially those cte^ing with reii#<Hts and pNioso- 
phical questicH^s. The latter discussions are often attempts by *e stuttents to work out 
their own values wid icteolmpes. (p. 231 ) 

Clearly, dating Is potentially one of tiie most im|wrtw»t laming modes for students. There are some who 
would argue that together with ail "non-arademic" activities they are not and shouki not be the lespooslbility rf 
educational planners. While few would suggest that the dating encounter should be heavily structured as a se • 
paced learning device (properly video-tajwd for later feedback to studCTt and tnstructorlK it does seem reasonable 
to acknowledge that the totality al the teaming that cot take piase » affected i»f the prevailing informal student 
(ami faculty) values and norrrK about the activity. For ex^nple, a rampus atnrri»phere dominated tiy tr«iitionai 
machismo and mannered male-female relations will result in far different kimte of s&id«it developrtient than will 
one which is characterized by eqi»lity, h<mesty, directness and caring. 

The n«»n point of thew two exampi®, bull sessior« and dstir^. Is to illustrate that «ie out-of-d^ activity 
of students has a direct relationship to the in-dass learning. If In the preparation of instn«tl«tal crf^ives for 
courses, faculty are not cognizant of the kinds of ^owth and development whidi can take place in other settings 
and of the relatiwship of that le«»ning not only to curricular otMectivw but to student need satisfaction, their 
tlesign of new instructional technologies will t» deficient. More, it may even have a negative effect on the <^lity 
of campus life in which it is introduced. As Janowltz and Street (19K) remark about educatbnal television. 

Rarely (if ever) is the CTUcial question asked: VW»at is tfie impact of television instruction upon 
the teadiing staff, the studait txxiy, arKi the whde organizational capacity of ihs educational 
institutiw? What are ^e effects m socialization of television instruction? In ^rt. what are the 
side effects - when the side effects may be more powerful tfian the direct effects. 
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Kteasurement of tfw Environment 

Those interwted in experimenting with new instruaional technologies are quite naturally concerned with 
measurmg the effects ot their experimental treatments. The hypothesis of this pai^r is that such measurement 
must mdude not only cognitive changes in students, Ixit their affective development and the nature of the 
enviriKtment which supports it. 

« u J*^'* ^ PfoWems. New instructional technofo^es in bounded, conventional systems 

o h^r education commonly take as their time frame fbr measurement the semester or charter - i.e. the urtit 
of t.me m which a "course" is offered. In the evaluations, pre- and post-coun» testing of treatment and control 
youps ,s performed in order to detemiine whether the course material has been acquired with^ater efficiency. 
Kare^^ are efforts made to retest at later times, except perhaps by test^kers comxmed with increasing the 
rehabihties of their scales. Seldom, too, are measures taken of the long-term effects on the thou^t processes or 
behavior of the student or of the ways in whteh his new knowledge has m has not been integrated into his 
personal value system. There is little research «4iich shows, for example, correlations between achievement scores 
m courses ustngnew instructional technologic and changes in personality growth and development taken at 
significant points in the student's colle^ career. In short, evaluations of new instructic^ t^nologies do not 
usually look beyond the course as the time unit of me»uren»oi. and they rarely make coor«ctions between the 
obiectives of the course and the broader aims and objectives of curriculum and stucfent <cf. Bom et al., 19721. 

Nor does the research to date seem concerned with tits relationships betwreen new technologies and the 
"press" of the campus environment. Fortunately, some consic^rable research has been reported on the nature 
T^"" '^^^'^"^'P* incoming student characteristics and the effect of the college environment on them 

(Astin, 1968: Feldman and Newcomb. \Qmh though critics of this kind of research are also becoming more 
vociferous (Feldman, 1972). In gro^y oversimplified di^ammatic form, the figure below shows some of the 
difficulty in measuring the impaa of new instructional tedmologies: 
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The figure reveals how the new instructional technologies are set in a context of a total environmental press, 
both affecting and being affected by it. Students entering the institution help (tetermine the press and are affected 
differentially both by the press and by the new instmctional technotogies which they may undergo. 

It will be helpfull at this point to illustrate how me»ures of the can^s environnwnt can be relate to evsloa- 
tions of new instructional technologies. Since the early anecdotal-type research of Boroff {19S8> and Jacob (19S7) 
many instruments have been developed to provide valid and reliable information about ooHege campus. Apf^oac^U 
to the study of colle^ environments vary. One is the method of perceptual analysis, in which varioia constituencies 
on a campus indicate the de^ee to which they feel activities of one kind or another are present in varying degmes 
(Stem. 1970; Pace, 1972; Pervin, 1967; Oiickering, 1970). The several insirumems used by the American College 
Tffitmg Service and the Educational Testing Service {e.g.. Peterson et al.. 1970) are perhaps the most widely known 
and readily available of this type, partly because the scales have been well tested and natiocuHly normed. Through 
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these questionnaires, it » piKSibte to cjbtain a picture of a camjHis in letmt of its relative emfrfiasis on certain cpaii, 
its efficacy in functioning and tfw quality of its environment measured by per ceiwd characteristics such as scholar- 
ship, awareness, community. fKoprieiy. practicality, quality of teaching and campus morale. 

Another apfWO»:h to the study of environments is tfw stimulus approach (Astin & HollarKJ, 1^1 ; Astin, t968; 
pace, 1972i. Here, member* of the campus are asked to descnk^ their own activities, and/or researchers ascertain 
thedegree of proiencs or absencc-of key «iv>ronmentart features (e.g., numbc. of fJKuHy. nun*er of books in the 
liiwary t Still a third, but n^ated. avenue of investigation is the depiction of the diaratxer of an enwironnwnt by 
describing the personalnv characteristics of the individuals who comprise it. Thus far, this approach has been 
developed primarily for students. Using one such instrument. The Omnibus f»ersonality Inventory (Heist & Yonge, 
1 968) it is fwssible to desc i an- a camfws in temts of its students' t^spositipns toward intellectuality, autonomous 
action, intr.werston extKJv.^t^'Oh. estheticism, impul«» expression and altruism, aiKl the d^ee to which they are 
personally iftt^^tet}, i«J!.>u«sly liberal and oriented toward pr«»ical mattens. Tbe instrument is ^sp of value in 
allowing tong«uu:..^'«»*c:ssments of the student body, thus permitting evaluations of grtwwth and development 
resulting (prcsumabiyjt from exposure to the campt« or selected portions of it <e.g., different majo" o*^ alternative 
residential arrangements}. 

Each of th^ approadies can be of use in the evaluation of new instructional technologies. A^in, If the 
assumption is made that there is an interactive effect anwng the technology, the environment and student growth 
and development, then inquiries can iw made of the nature of the relationships. If the designers of experiments 
using the new technolo^es include in their planning the notice that import»>t, loog lasting Uberrt education 
requires supportive environments, then it siiouWI be possible to control for tfw environment in the evaluation. For 
example, selection of cmain ttormitories for acthiities suppotti«^ of in-class technologies can be ma<te. Assess 
ment of tfie quaMty of the dormitory environment compared with control groups can be made before and after 
the introduction of the new technology. Similarly, student personality dispositions can be ascertained in these con 
trolled envirunments, permitting later testing for ctian^ 

Beyomi the measuren^ problems in evaluating the effects of new instructicMiai tecdtnologies on students are 
a whole host of other environmental factors particularly those involving faculty. This is especially true on bounded, 
conventional campuses. Faculty in the social setting of a traditional institution are not generally Inclined nor 
encouraged to share their teadiing n^ttrods, problems and successes with oalleac^es. It Is somewhat surprising, 
then, thai new instructional technologies are not looked on more favorably, since as noted eaHier, the assumptions 
and objectives of each are actually quite simitar. The problem is comi»unded when, or if, tiie broader set of 
objectives sug^ted for new technologies is proposed. Faculty on these campus will resist not only new tedi- 
nological features (Evans & Leppmann. 1967) but the expanded set of objectives and the nej»ssitv of environmental 
support for them. 

To summarize, learning v^ich affects the whole f^son requires careful desi^ of the total environment in 
which the process takes place. In bounded conventional systems, it is unlikely that new instructtonal technologies 
will result in profourel tMnefits without positive social system reinforcement. The ^»em» of reinforciiHi con- 
tin^ncies in dormitwies and other student -ctominated setting results in a narrowed intellectual fcwus and a 
*ortened time dimension into vrfiich n&fii material is inteyated. Students, are ted, in other words, to view their 
current academic experience as a shortrun hurdle to be conquered and quickly forgotten, or perhaps stored in 
memory for some as yet unreco^ized future need. 

Measurement of the attitudes of those in the environment in which new instructional techndogtes are 
introduced will reveal the presence or absence of values which support long-lasting l^mii^ and which encourage 
attention to stiKient growth and development. The bifwcation of institutional c^jectiv^s (which on traditional 
campuses are geared primarily toward preparation of the student as a future citizen) and individual student needs 
(which are temporaHy s^tuate^^ m the present) ts accentuated by new instructional technologies which tend to 
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neglect the »atter Only when those concttrrted with innovative teaching and learnrng methods incorporate a 
broader s«t of educational objectives will the need for support.ve campus environments become of sufficient 
•mpoMaiice to iM^oduce them. Jn sum. attention to the sociology of learning as well as the technoJogy of teaching 
in the hght of a comprehensive set of institutional objectives is needetf for a truly successful educationaf 
entetiirise. ■ ■ ■ ; 
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THE COMPUTER AND THiE ^OURTH REVOLUTION 

Or. Andrew R. Moinar* 
Prp^am Direct<M^ for 
Applications and Courseware, T.I.E. 
Ntational Science Foundation 

I. THE FOURTH ftEVOtUTICW 

The Cariiegte Commission ort Higher Education has raw 
Revolution" which takes its title from Eric Ashby's dt»ervalion th^it four yeat revotutions have been taken 
place in education,' 

The first revolution was the diff^entiaition of adult roles so that the task of education for the young was 
shifted in part from parents to teachers and from ^e home to the school. 

The second revplutmn was the adoption of the written word as a toot of education and with some reluctance 
writing was permitted to co^exist widi the spoken word. 

The third $i^ff{cant change was the invention of printing and the widespread avaiiat^itity of books. 

And the fourth revolution is the development of etectrcmics, notably radio, television and the computer. 
The computer, however, is the imperative in the fourth revoluticm. 

II. THEGR€)WTHOFCX>MFUTfNGINEOlfCATIOia 

In a little over 15 years, academic computing has grown at a phenomenal rate such that in hij^er education: 2 
Access to Computers 

AH major universities and most colleges provide computing services to their students either throush local 
taciltties or access to off-campus fac<lii.es. 

Expenditures 

Expenditures m the last decade have increased over ten told with million dollars spent on a>mputing in 
1963 arKi an estimated 540 million dollars being sf^nt in 1972. 

Type of Computing 

Of that amount spent, 30% was spent for instructional use; 32% for research use; 34% for administration and 
4% for services to other institutions. 

*Th« v^e^ an^ those of xho author 9rti$ do not nscefsarHy rsprefcmt tho9e of the Ntttktfi^ Sckmca foundation. Thtit {Taper 
ifl bas«id on commonts ado a. tt^ Apfii 17, 1973 Annual Convention of The A»M>c:t4tion for EcKjcatiorml D«f4iSyttenii m Nwv 
Offaans, Louisidna, and further Hlustrsfed at fhe Educationdl T^^nofogy Syhnposkim atSto.ty Brook^ Maw Vorfc. 

^ The Fourth RevcMution; Instructional TeGhnolo^y in H^har Education. A r<¥»ort and r^ommendattons by The Carnegie 
Commtssion on Hii|her Educistion. (New York. McGraw-HHI Soofc Company, 1^721. 

^iohn HsfTtbien,' Inventory of Corriputers in U.S. H^lhar Education, 1969 1970. (Atlanta, Georgta: Soutf^ern Regicwiat 
Educatimi Board, 1972 K ? 
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Sources of Support 

This growth has occurred in spite of a diminishing percentage of contributions from Federal sources. 
Currently. 70% of expenditures for computing comes from institutional funds. In 1965, 36% of the funds came 
from Federal sources; in 1967, 23%; and only 1 7% m 1970. 

Computer Science Maiors 

In the last five years the number of computer scienct; degree majors have increased by a factor of 15 from 
approximately 5,000 to 75,'X)0 students. 

Language Substitute 

Four hundred and seventy departments now permit the substitution of computer languages for foreign 
languages. This is a three fold increase in the last two years, 

Curricula 

Computer-based curricula has grown from less than 100 computer-based packages in 1965 to over 6.000 in 
1972 - an increase of a factor of 60 over a seven year period. 

In secondary education while the figures are less complete, we do know that in 1970 34% of the Nation's 
secondary schools had access to a computer for administration and/or instructional purposes.3 In 1963, a survey 
estimated only one percent of our Nation's secondary schools used the computer for instructional purposes; m 
1966 that increased to 1 .7 percent and finally to 1 2.9 percent in 1970. This represents a growth in instructional 
uses of over 1 2 fold in little over seven years. This amazing exponential growth of academic computing has come 
about in spite of a growing financial problem faced by education. 

III. NATIONAL SCIENCE FOUNDATION PROGRAMS 

Dr H Guyford Stever, Director of the National Science Foundation (NSF I. says that the Foundation's 
programs provide for continued strong support for science with an effective balance between resear«^ and 
education programs.^ The higher priority, however, will be research programs. The NSF budget for F'^cal Year 
IFYI 1972 was 600.7 million dollars and the FY 1 973 expenditures are estimated at 615 million dollars^ The 
requested FY 1974 budget of 641.5 million dollars includes 58.9 million dollars of FY 1973 carryover funds. 

The new restructural Science Education Imorovement Programs provide support for programs that are 
designed to (1 ) increase science education effectiveness; <2) help provide the essential number and variety of 
trained scientists and engineers and; (3) make the general public more knowledgeable as to the potential uses 
of science and its limitations so that they can deal more effectively with problems requiring an understanding of 
science and technology. NSF Science Education Improvement activities in FY ;^t''r«r'rriVR miZ 
47 0 million dollars versus the planned 80.8 million dollars and compared to a FY 1968 high of 125 million 
dollars, in F Y 1972. the budget was 73.4 million dollars. The requested FY 1974 budyst is for 60 million dollars. 

The Office of Computing Activities 

The Of.ce of Computmg Activities was established in July, 1967 tc provide Federal leadership in exploring 
and developing computer technology and th. uses of the computer. The Office administers a program in Computer 
Sfentld Engineering which supports basic research in Theoretical Comp-ter Science, Software and Programming 

3cnac.>^ A. D arby. Ar.hu, L. Korot.m. Tanla Rom.,hko. "A S.rwey of Computing Act.v,.., ,n Secondary School.." 

(Wwhtngton, D.C, Amancan Institutei for Hetamrch. 1970) 

''Nfl'.onal Science Foundat.on, BurtaBt Highlights for Fiscal Year 1974. 
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Sv^ems ami Cpmpuier System Desityi. Another program computer Applications In Re^rdi, seeks to su«>ort 
res^rch sf inltes which focus on t^^ explqftation of advances in computer terfrnology to further research in 
science. Support is being provided for expforatory studies to devefop a National Network of computer-based 
tesouf ces <n support of research and educatw. A new pr^ram. Computer Impact w Society, is concerned with 
studies of Ihe impact of computers on organ iirat ions and iodividuats. The FY 1972 buc^t for OCA was !2,§ 
miHton doftars. In FY 1973, the Office wHf spend TOO miHron dollars and tO.O miriion dollars is requested for 
FY 1974. 

Technofo^cal InoOYation in Education 



The Cornputt^f Imiovation in Edoc^tton program has tieen transferred out of the Office of Computing 
Activities to the Education Directorate of the Foundation and renamed the Tedinoloffi^l Innovation in 
Education Group. The objectives of tWs youp are to explore and cfeyelc^ new innovatiw uses of the comfxiter 
and relateti tf»fT^'.oloc^ies in education. In FY 1972, 8.4 milliofi dollars was spent for this activity and 6.0 million 
do«a»s ts estmiaicd for FY 1973. In FY 1974, 7.0 million dollars is requested for the expanded functions, t 
would like to briefly describe some of the activities oirrently t^ing supported. 

IV. CCNWnJTER-BASED SYSTEMS 

We have supp^jrted the development of a wide variety of instructional systems. Oregon State University has 
developed an on line, interactive, gt^fjhic system for ctassroo^n use in scieiK». TTie system permits ttte instiuctor 
to dynamically control the graphics ternHnal with a joystick input device, A large screen television pro^tw 
enables the instructor to display the comptttcr output on a 6 foot by 8 foot screen which Is dearly visible to a 
class of 2tW students. This classroom use of a sophtstrcated terminal sr^sficantty reckic^ the cost per-student 
hour of instruction.-^ 

Project IMPRESS at Dartmouth Ck»flege is a com^rsational^ interactive packa^ for the social sciences.^ 
It permits the student to have access to o«er 1(K) files of «)ctal science data a variety of topics. Rather than 
hear a lecture, the student formulates hypotheses about current social problems and tests them using a variety of 
statistical psicka^s using c^rwt social data. The system includes f^agogical rixirir^ that guide a noviro throufi^ 
complex statistical proyarrts in a conversational manner with minimal previoiB experience ar^ permits under- 
graduates to r>erform learning activities which would not otherwise be intrcKluced until later years. This aj^oach 
is unlike CAI and yet different from traditional prc^lem-solving. The potential for IMPRESS and m^ advwced 
systems to retfiare costs aid offer a m?w instructicmal forrrvat is very aitractive. ^ 

Currently, Or. feymour Paf^rt of MIT and Dr. Robert Davis at Syracuse University, now with PLATO, have 
demonstrated the use of LOGO, a con^iter langua^, to wd assist fifth ^ade chiWrw to compute functions 
and soive chaculus like problems. This activity seeks to avoid the rigid sequential presentation of mathematical 
concepts and throust» the use of the comfxjter and computer artifacts introduce hi^er order concepts at a much 
earlier age. With this «v tern children t^ave demonstrated their ability to write pro-ams to draw simple figures, 
construct complex geometric desiyis, write a frame-by f rame movie, ^nerate music, write computer generated 
poetry, solve motion problems in physics and program the movements of a marionette.? 

NSF has developed a number of exfjerimental models of com|;»jter networks for developing and sharing 
computer netvwrks for developing and sharing computer resources for instructional purposes. Some use batch 
processing, others are interactive. One network confines dosed^ircuit tetevisic^ with remote computing. One 
expenmpnt involved a nationwide disciplinary network for Chemistry departments while still another us«^ a 
network of minicomputers for small colleges. In the past four years, some 30 re^oml computing networks were 

^T*m G. Kellsy et "infersctive Classroom Orapfttc." Orevon State Uni^/tfntiy, 1972. 

^J*wies A. Davis. Using the IMPRESS System to Toat h So^tology. In "Proceeding of tho Second Annual Conference on 
Comf utef$ Jn the UntiBrQt&tiuate Ciirr»cul«/' (Hanover, N.M,, Dartmouth CoHo^, 1971 > 

''Seymour Pap^t, "Teaching ChtUirCf, Thinking," IF IP Work! Conference on Coniputer Education, August, t970 and 
••Twenty Thing* To Do with a ComiJulflr^/' Art if *^ lal lnteni0ence toboratory^ Masa^husetts Institute of Technoloi^. Cemtirldoe 
Messorhijfiett» 1971. 
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^abiishect tavoiving TXXi instHutiCKis of hi^r education ami some secondary srfioots* The program tia$ provided 
merr^scirs i^itti access to a larger rofnputer with a tibrarV of instrcRtionat and prcMem-MiivtfM9[ programs d$ a$ 
technicd! assistance and training. Today, approxtntatefy thrw quarters of the networks are^f-sustalning and 
continue to operate after the terminatioi* i3ff Federat fumis* This hl^ty successful exiwimental prt^am is now 
being phased out. ^ 

. ' . ' , ' . ■ ' ♦ 

The research and cteyefoinnem of the 19^% althotigh sewreiy limited by tedtnc^ogy and p^gc^, 
(temonstrated that students could l^m i^M^ing CAf at least as well and in some case$ better thm by more 
traditional methods. However, the equipment l^ked grafrfiics and audio cap^ility. Software systcnm we^ 
not flexible or easy to (^0{p^am and v^re mud) too costly - fncbaMy by a ^itor of tw. In order to remedy this 
situation NSF Sfxinsored the deveic^ment ar^ddemonstraticm of tvra advs^ncid systeim; one following a 
centraiued, utility ^iproach an^another foik>wing d modular or c^rcentralijml apfKoadr* The profess will take 
five years and ten mittion dollars of NSF support. 

m . . 

The University of Illinois PtATO IV ((^o^pa^ir^ Logic for Autonr^tie T^diing Opi^^ionsKsystem is 
a>mrolled by a Control Data Corporation &XKi series a>m{xiter with up to 1,(K)0 terminals connected to the 
computer ihrou^i a single television channel*^ The terminal consists of a key^i ami a y:4dsnr^ display ft^ ixmputer 
generated informatics. The plasma display panel omsists of two ^i^U of ^parated by a thin layer of 
Points on the display are fired by the comfUtter to^r^ate words« figure ^ drawing, Lfp to 2S6 colored prc^ 
recorded microfiche slides can also be displayed on the screen as well as dynamic information superimposed over 
the static display. More than 4^000 random a<xess audio messages of up to 21 minutes in length can tm recc^nted 
and selectively presented under computer control. 

A touch sensitive panel permits a child is resKling and doesn't realize, for example, the word "elef^nt' 
to touch the word on the display panel and have a picture of an eteprfiant appear and hear the word in his ear- 
frfiones. The system will provicte courses in elementary and secondary school currioila as well as community colle^ 
and university level courses. 

One hundred and fit4y terminals are up and operating at Urbana and 40ChS(K) termi^mts wiH fc» installed ittm 
Urbana to Chicago by Sep^en^r of thts year. The system was ^so ^monstrated via sat^ite in Italy and Spacer- 
land last summer and the Organiication for Economic Ohop^atkm and C^lqpment iO£CO| now has a terminal 
in Paris. 

The TICCIT {T\rne l^ared Inter^hw Computer Ccmtroii^ fnformatifm TeteviskmJ system is c^^ated by a 
Data General, solid state, minicomfKiter with 128 student termii^s.^ The terminsri c»r^h;ts of a keybwrd, a small 
color television to display computer ^nerated displays, victeo tape or slides and audio, l^e tx^ui^m^re will ccmsist 
of two years of Mathematics ^d English and will be written to follcwv a prf^ited set of design {Ko^ur^. The 
intent is to design and validate proi^ures that will yield hi^ quality cotir^ware whidi will be u^rful to tar^ 
numbers of students in standard courses throu^out the country, Tf^ system will Installed and tested In 1974 
at tM> community cotle^ — Nortt^n Virginia Dmimunity College and Martoipa Community Coile^ in 
Wioenix, Arizona. 

The Education Testing Service of Princeton, Jer^y will evaluate and re(x>rt their findir^ for both the 
PLATO and TICCIT project. 



^Ponsld L. Bitter and D. Skm>0rfi^. "The C^sIgn of an Economk^Uy Viable targe^Scala Computer Bas^ tducation^ 
SyMem/' pp. 14-34 In Computers \n InMruction, <E<1,| B. 6. tevl#n, Raw! ConporatkMi, Santa Mofi4ca, CaUforfiia, 1971. 

^Kenneth J. Stetten. ''The T#chnolo«y of Small LcNiei facilities for ln«tructional Ui»/' pp. 3Mt. In Computers In imtrw 
tlon« (Ed. I H. B. tevlen. Rank Cofporatfon, Santa Monk:a, Ca^ifof nia, 1971. 
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The MMRE Corporation i% currentlv (terrKMis^tr^ting the fea$ibUity of mteractiw, comfRiter^niroile^ 
television systems for home use, They have coupled a cable tefevision system with « ccmiputer to demonstrate 
the feastbitity of using a standard tetevision receiver for home Computer driven displays for hwnes in the new 
town of Re&ton, Virginia. Cabre television f^rmits twoway conferencing, Tfie home user communicates with 
the computer throu^ a toudi t€K»e te(ej*ione, tenons are addre^^ed directly to his television receiwr and he 
may use his touchstone phone fof calculations or ansiwring questions he ctemcmstraticm incluttes the home use 
of 27 lessons for fourih gr^le arithmetic drill t^^ice. A simple Computing laiguage, Mr, Computet, is also 
avaitabh^ for writing student prigii^ted prc^ams. $ince cable television f^rmlts two-way conferencing, the 
teacher may work directly with students at home during th^ cfay <^ conduct adutt eduction in the evening. 

It is foreseen that 2,(W) individual homes may served for a nominal mcKi^ty fc^. The home user will be 
able to shop remotely by televisiwi. He will be able to revive his mail automatically dialing his pose office 
box number. His salary v^ill automatically deposited in his bank account by Ws employer and he will be 
notif ie<l thrcMis^ his teleyii^icm set. Upim his authorization, bills such ^ mort^Mige payments will autcMnatically be 
trwsferred to his creditors. The compui»r controlled television will display this information and may take a 
polaroid snapshot for a hard copy receipt. The system cai provicfe fire ami i^curity nwiitoring- The user will be 
able to interact directly with his doctor and thereby reduce the number of office calls or home visits. He will also 
be ^le to make h<me movies using a smalt Iw -cost video camera and later play it back throuifi his home 
receiver. Demonstrations of these service are currently being concfcictc^ at Reston, Virginia. 

V. COURSEWARE AND TRANSPORTABlLtTY 

Based upon a numt^er of NSF initiated studi^^ it became cle^ that coui eware was a critical problem, 
EOUCOW, using 35 experts in a Delpftic seminar, concluded that the most citical factor inhibiting the use of 
instructional computing was not cost but the lack of good, readily available computer-based educational 
materials.tt This led us to initiate a mor« purposeful attack upon both die short-range and long-ran^ solutions to 
courseware development and transportability prdblems. 

First, we initiated a statc-of-the art study to determ* what existed and how useful t is. Second, for the 
short run, we emphasised the development, documentation and testing of existing, but yet unput)lished packages. 
Third, for the long run, we supported the deveic^mem and analysis of experin^ntal moctets for creating incentives 
and transportability in order to tetter understand die characteristics of this rapidly chiHiging field. We haw 
encoura^ interdisciplinary c^ferem^ for diffusic^ of compuier-b^d instruction through hands-cm experience 
and face*to-fa^ discussion, with curriculum develo^rs. Finally, wa continue to dei^tc^ computer courseware for 
classroom use. 

&ra^^ for Curricular Devdiqmumt 

T>ie Hun«n Resources Research Organization IHumRRO) of Alexandria, Virginia is conducting a state of 
the art survey identifying all computer-based curricula by discipline, level, lan^age, machine, and instructional use,t2 
Compelling examples of computer*b^ed instructicm and associated costs of development haw been identified. 
Surveys of publishers and vendors v/ere made. Currently, alternative strategies, thetr associated costs and conse- 
quences are being evaluate. One recommerxiation HumRRO has made is the a^eation of a National cl^rin^iouse for 
computer-based materials for hi^er education. 

^ ^Kenneth J. Stetten and Rodnsy K, Lay. "A Study of th« Tecfinica! and Economic Con^kisfaiions Attendant on fha 
Mom« DaMvary of Instructional and Other S^ciaKy Ratatad Sarvicas Via Interacti^ Cafola TV," MITR£ Corporation, McLean, 
Vfr^ti%m, Oacambar, 1972. 

'^Ernest J, Anattasto and Judith S. Morgan, "Factors tnhHjiting tha Use of Computef* In f nttruction/' (tOUCOM: 
Princeton, Naw iar «ay, 19721, 

^^Robert Sofdei. at al., "A Study of Comptitar Ba^ed Curricula/' Humim Ro^urcod Rosrarch Organlzatloo, Alsxandria, 
^Virginia, 1971. 

^3jo»aph R, Dank, "CONDUfT A Concrata Pipaiina for Softwara^Sfarvad Lfttle People/' in tha "Proceedings^ of iha 
1972 Confaranca on Computers in tha Undar|p'a<laat)a Curriciifa/' Atlanta, Qaorgia' Southam Ragtonal EducatH>n{ii Board, 1972. 

4 
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CmtmAum Oe^diopiTOnt — Profocl GOMIHiTe 

. Based upon our survey of publishers, it bei^me apparent tfoat tftey were uninterested in <tewl tiding 
publishing cominiter-bised modules. Also, the most ccNnpriling materiats used in classropms were fouiKl to be 
unducumented and only tocaliy available. Further, netther tnstitutionat ineeRtives, as rekr^ time^, nor 
royalties were available tostinuiiate indivtduats to ctocument^ dtebugand lest th^r programt. Through a grant 
with Oartmourti Colte^ a national s^rch was mackf for e)cemptdry materials in the envircmmenttf ami related 
sciences, Th^ authors were invited to Hanover for title summer oni the CoiiejEfs provided onnputer time, 
p^o^amming and edttori ai services. Standard formats were ctevised for stud^t, teacher contputer 
program materials. fMiMshers are being ccmtacted atKAit publi^ing an ^^it^ ^ies as a padu^* Throi^ 
special arrangentents« both the authors md £>artmouth may receive rc^alti^. The ^pro|^ we h<^ i^tl 
accelerate the mjmber of c^Hty psK:ka{^ avallaMe and could create a comn^rdai nKKiel for citriculum 
deveio|)ment efforts which the Cdlletg^ could pursue after the terminaticm of the grmit. 

TrmspoftatriHty Pri>l^ - C<H4D^^ 

One of the most 5eri<HJs obstacles that we face is ^ f^roblem of tr^^»(KtabiHty of progFems, Pn^ms 
that run at one location are rarely availiMe cm reqt^st and if avail^ie^ seldom woNrk elsewhere* Om sotutk>n is 
a National Academic Network which f»rmits itw field to develop but also permits others to have access to the 
materials- ^ort of intercwinection. however, st^mcbrcb nnjst te i^opted to make materiais compatible for 8 
variety of existing systems and madiines. 

CONOUtT is 3 consortium of five regicmal networks inv<rfving lOT colleges «id imiversities with an enrdt- 
ment of ^.OK) students.'^i It seeks low-cost dilutions to the pr<^em of ^^sportabiiity. Dtsciplinary 
committees have identified 100 f»::kage$ in eig^t disciplii^ry an»s. CONDUtT<!entral in ocH>peration witti the 
computer center director, and curriculum coc^natc^ in eat^i of the five networks hSw dooimented and 
certified the instructicmat packages^ estiribHs^ed is^ocedure^ ami €^ms for transporting the mateHds ^ created 
self Wnstructionai video taped materials based upon workshop expericfic^ and are moving the materials frwi 
center to center and ultimately Into the cl^sroom for use and evaluation. An independent or^ization, 
HumBRO, has devised numerotR hypotfie^ cwceming organizati^mat structure, trai^MKtebitjty, information 
needs, training and ao^nting prowdures and will evaluate ^ entire system. 

Confef€»Kes 

The di^mination of infcM'mation o^H^mlng conHRiter-fa^ed oirricuta. Is, at tiest, very dlfflailt. From 
experience, one of the most successful mech^isms for rapid dissemination has been throu^ ami^ multi- 
disciplinary and discifrfinary inferences at iMiich users present arrf tryout <hi comp. ter termirmte a wide variety 
of materials. The anrru^ CcHifc^ence on Oon^Kiters in the Umtergradt^te CkMricuta prodtioes 60 to 80 ptMi^^ied 
paters on computer applications in approximately 20 discipline and attr^^ approximately 1,(H10 attendees. 
This y^r's conference ¥vill ^e held ^ne 18-21 at Claremont Colleges, Ponwna, Calif wite. Disciplinary 
Gonfej^ces have been held in Physics, Chemistry, »id the Humanities. 

Courseware ^ 

The NSF Scieni^ Course ImyKovement f^c^m is currently supporting a four year project on Computer- 
Basmi Education (C BE) at the University of Texas whidi is aimcsi at evaluating the imp^ of a critical m^ of 
computer-based aJucation on a bro^ range of acactemic subj^;ts. Courseware wilt be developed in approxirmitely 
16 interrelated academic discijrfines. The Foundation is providing 1.3 million dollars and the Uniwrsity is 
providing a like amoi«nt to develop ami use the cot^rseware in the d^room. 

^^joMph n. t^k, 'XONDiJIT - A Concreta PifM»4in6 for Softwara^Starvsf^ LHt\9 P&jpXq/' m the "PTO<;mfd\ngs of the 
1972 Confitrertca on Computm in the Und^rgrsduste Curricula/' Atlanta, Georgia: Southern ne^onsf Educatloftal Board, 1972. 
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VI. COMPUTER LITERACY 

AppfOKtmiitely 60% af the war Itf's computers are used in the United States, The Unit^ States is the prime 
user ancf principal p'-wluc^r of carn^^iuters ami compu of cumputers ar^ approaching 

one bf Iff oil doHars and with respect to bi^lance of payments, the United States is the only net exporter of computers 
in the vi^ortd. A recent National Bureau of Standards study found apiiKoxiinateiy 2,300 different appHcations of 
computing in business and industry. The computer is fast becoming a national basic industry. 

While the printed test was important to civilisation because it permitted man to extend his imrnediate 
memory and to accumulate his past exparietKres, its true impact or> scKiiety was not reaUzed until there was a 
literate populace. 

So It is with computers. The comiHiter in its short span of history has been of>e of the most significant 
dmpltfa»rs of productivity. However, the widespread use of this powerful tool may be limited by the lack of 
public understanding and the public's rnabif ity to see how the computer may benefit them and s<M:iety. 

A lecent AFIPS-Time Magazine study, ''National Survey of PubHc Attitudes Toward Computing,** showed 
strong public anxiety toward the computer, i* Approximately one third of the Nation's adults still believe that the 
computer is some kind of "thinking machirre*^ and has the power to think for itself. More than half believe that 
they are too dependent on the computer and that computers are changing their lives too rapidly. Fifteen percent 
believed that their lives are worse because of computers; this is twice the numbcN^ who feel technology in ^eral 
has made lite worse. 

Federal programs in science and technology aimed at making sispiificant gains in productivity and the 
general well-being of the public must also be coitcerned with public understanding of these developments. An 
informed populace a^n better understand the strengths and limitations of new t^hnofogical innovations such as 
computing, ^he degree of "computer literacy" among the ^neral iK^pulace may be the limiting factor in rapid 
technoiogtcal advances and with it, national productivity. 

A recent report lo the National Science Foundation by the Conference &>ard of Mathematical Sciences 
(CBMSh entitled "Computers in Hi^ School Education/' has also strongly recommended the development of a 
"computer literacy" curriculum for seccmdary schoolsJ^ They recommended specifically the preparation of a 
funiof htgli school course in computer literacy; materials for an introduction to coit^uting; materials for a number 
of science courses; special pro-ams for students with unusual aptitudes in comfxiting; vocational education; 
teacher training anti the creation of an information clearinghouse. 

Vlf . WHAT IS TO BE DONE 

While the fourth revolution is gaining ground, the current financial crisis in education offers a challencp and 
a major opportunity to advance the educational use of computers to a new plateau and in the process increase the 
quality of mstructioii while siyiificantly reducing the costs. However, much remains to be done. 

What we need is a complete system and a total curricula. Anything less will be costly and ineffective. At 
many locations, educators are still working with cast-off business machines and providing rigid one-dimensional 
instruction. Throuc^ thp persistence of a number of educational innovators, we are on the verge of a "first 
generation" computer-based system. When this is accomplished we will have a medium without a message. What 
is needed is a critical mass of quality materials — a total curricula. Then and only then will we be able to 
demonstrate significant cost-benefits. 

^^"A National S if vey of the f*tit?l)c's Attitudes TowurtJ Conjputefs/' (Nftw York: Time Magazine, limo Life Bmiritng. 
BorkeMltjr Confer, 1971). 

^^"R^ommenclatfons Regafflmci Comptilefs <n M»qh School f ducotion." iW^i^hlnqton, D C. Conf«r^ce Board on the 
Mdth«n">at«c*ii Scienter, 1072). 
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Marshall McLuhatt dr«}yes that theelectrcMiic revoiution »s fotaliy new^id is dwiging the very nati^^e of 
human jperception and e^per ieno^. tn the of tetevtsion and cQinputer, McLuhan says that w« move Into the 
world of pattern reas^ition and out of the world of mcfre data dassif icatlcm. We t^not irarrety convert old 
orugramrned teicts into computer formats. We must retool mir currtcufa to take adyanta^ of the nw degree 
of freedom the computer offers. This trend is already tecomingevictent in mudt of the new curricula. 

We need a market medianism to develop amt distribute m^^ 
the distrititition of computerfoased materiafs uninteresting. Equipment manufacturars, espi»:iaUy in the mini- 
computer fietd, f ind that the avatf ability of coniputer-based n^terials is invortant in the sale of equipm«it 
systems and they have done a good Job in j^iravidi^g simple progr^s and materiats. HtAvever, etjuipment is the 
manufacturer's lousiness - rtbt curricutar dKveloptrent . Therefore, unless there ate wune rrwpr nmg^ between 
publishing houses and vendors, vi^e are dulikriy to see the commercial sector prfay a vital tok in echication. 
Education will have to create its own ^r^j^trcKiat mechanisms {^mcificatty designed to create, develop arKi 
dissemir^te curricular materials or face the pro^ieet of incoming the wwkt's largest ami most exp^iw 
cottage industry. 

We need a new orc^tzattonat medwnism to handle the new teciinology. iohn Gardr^r cmoeobset^ that 
HKMt wganiations have a structure tl^t was d^r^ed to soli^ problems that no longer exist, Vtfa nnust go beyond 
current arrangements if we are to develop. There are still no incentives to write progr«fi$ on courseware v^ich 
upgracte the quality of education or reduce the cost-per-student hour of instruction. Writers seldom get royalti^ 
or promotions and seldom $hare in any ojst saving whidr may accrii^ to the University, instructors are better 
advised to write researcti papers in their discipline than to develop computer coin^eware. Educational 
institutions wfio are serious about cost-effectiveness wilt evemually haw to devise institutional rewards and/or 
monetary incentives if they wish to reap the t^nefits of instructic^al tedinology. 

While there is a revofut^cKi in technology, there is an even greater rsiraf ution taking (irface in ecliHzdtioa 
Just because we are on the ri^| track doesn't mean that we will not be run over fay changing events. Obsolescence 
is bei^ming a major problem in our knowled^^ sodety . The avera^ man in the wwk force can be expected to 
change |obs six or seven times in his lifetime. Tbe half-life of professional man is only sbmt ten years. 

P^ul Arrner proposes the "f^ul Principle'* whidi he contrasts with the "Peter Prindpte" - irKlivfduats tend 
to rise in organizations to their level incompetence, The "Paul Principle" states that Itidividtiais oftm become 
incompetent over time because they become uneducated or obsolete. He says that hi^ier education is not even 
remotely prepared to take on the C(»itinuing education as a major task. He su^ests that unh^ities of tfie 
future wilt operate somewhat like hc^tals. They would admit students not |ust one or two times a y^r but 
continually. Oi arrival a dia^osis wouM be made of the defic^d^ in the sti^tent's knowi^lge mnS the 
educational process of individualized instruction would be designed to fill the gaps. Modern tedinology is 
now capable of delivering instruction to an individual's home, place of employment or any other location. Educa- 
tion is no longer restricted to the dassroom and must seek a new rote in the knc^vied^ society. 

What is the Federal role in these developments? While there is no explicit federsA policy toward computing and 
while it is difficult to identify prc^rams with computing in their title, there aie many activities being initiated at 
the fmiera\ level. The Federal program^ tend to favor innov<^tion and research and tend to iook at technology as a 
v^iie rather than at computers in particular. Currently, programs tend to focus cai increasing fvcxiuctivity^ that is, 
improving quality while redudng costs. They seek to su|;^rt projects wh»:h benefit many institutions and many 
disciplines. Proyams of institutional supfx>rt, equipment grants and support for the acqutsition and upgrading of 
computers have b^n greatly reduced or completely eliminated. The acc^isition and development of tedmology 
and systems will fall to the States and to the education institutions involved. 

How will this aff^ the future of computmg? My prediction is that die development of instructic^al computing 
will follow the Xerox mcxtel. That is, in spite of the fact that the initial dry prcK^i^ ospying machine was mwe 
expensive than the wet proc^sing machines, ft was adopted bacause of its convenience and its ability to meet a 

'6p*Ml Afm«f , "Tn» Paul Prmeipia When Terhnofogy Outgrow* Msn/' Q«r<atfic6. 24, No, 8, Dec,, 1970, ftp. 29^30« 34. 
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compelling need. Similarly, computing is so compelling a tool that it cannot be stopped. How fast it will take 
develop antl how oxpensive the <levelopment will t>e are the only questions to be answered in the fourth 
ff'volution. 
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THE TICCIT PROJECT: 



DESIGN STRATEGY FOR EDIKJATIONAL INNOVATION 



C. Victor Buruiefson 
Director of Computer Us« in Education 
Bri^am Young Uni^rsity 



The term TICCIT refers to the MITRE Oirporattan's "Time-shared inter«:tive ConHHiter Ccmtroiled Infor- 
mation Tetevi$ion," a hardvware ctmcept eniiJlc^ins con¥ent)on8l »!fevai(^ and caWe teehm^ofies and minicom- 
puters. Being derived f r«n a hdrdware descriptitm, ttw term does not si^^ wrfwt we at &ri^m Young Untvenity 
regard as the educational contributiwis of the proiecL The puriKKe of this paiier is to fAace the hardware contri- 
butions of the MITRE Corporation in contend with liw educatirniaT omtributiofs «td coirteware tksigin strate^^ 
J, which have been evolving at Bngham Young University and to show how these strategies apply to tfie needs and 
spafs of the project, including th«e of teaching cwnmunity collsge En^tsh and n^thematics moro i^oductiveiy. 

Experience in c<mferen(» pres»nt3tion$, workshops and visitor CKisrtl^m to ttw TICXIT pro^t has proved 
to be difficult in conveyinft all of the €SMntial features of this prt^t to a mw audi^ce. The TICCIT {Mroject is not 
just ffliother conaputer-assisted instruction (CAt^ prcjiect.' it is a hopeful bat radical ^p«rture. Those fanuiiar wid» 
existing CAI projects may be daceived when tfwy attempt to af^ly familiar gener^izadtMis *«it hsHrdw»e. CAI 
languages, tutorial, drill, stimulation and other instructiofwt stratef^ra and utilization niotfels to TICXiT-for most 
of them do not a^V • Those who have seen sanHHes of "learner cwttrcT' must be prepared to reifcfine their 
MKlerst^ng of this concept w4ten disctnsing TICCIT, Taken alorte, each innovatiwi in har^ftare, software, course- 
ware and implementation seerm simple md ratural enough. f«>wever, one must perceive alt Jf th«w facets, working 
in harmony toward educational spals, to understand all of this "elephant." rather than hkf die blind men, some 
discrete part of it. 

mt 

Sofitd l«slory of th9 TICCIT fNt^^ 

If TICCIT IS indeed different. It {s different because of the disdpltned ctesij^^ abroach to the solution of 
educational prdt>tefns and tl^ interdhfdptlnary cross-f^iiiiation betmi^ the cfesigr^r^ mt6 de^c^iers at three 
tmtituticms, the MiTRE Corpi^atiOT, the Uns^rsity of Tex^ CAI taboratc^ end &i^m Youn^ University 
(BYU). The systerm engineers and computer scientist at the not-for-profit MITRE Corporaticm provided the 
knovv4iow in hardware and software de^lc^ment and inte^^lon. Unlike a maiufacturer of existing hardware, 
MiTRE was not bound by the cwrstraints of an existing prmhK:t line. Educfation for too Img iw hwi to make do 
with computer products designed for bi»in€^s and science and mltingly marketed to educators a& m instructional 
tool ctespite the inappro.oriateness of tiie hardware and software capj^lities. Unlike a laboratory at a single 
university, the system engineers at MITRE are exf^rienced at the prcxress of working backward from a problem 
to be solved in society at targe, to a concrete solution and seeing this solution all the way throu^ to system 
integration, installation and testing in a real-^mld environn^t. 

Aware of its own educational and social-science {imitations^ MITRE looked elsewhere for ^ant^en to specify 
educatmnal goals and strate^es. The Uruversity of Texas CAI L^ioratory, me of the of cfer and more successful 
of fts kind, contributed a learner c<Kitrol philosophy and an educational desi^ sci^ice appro<R:h. This included the 
idcntificatiofi of neecte, the specification of goals and the development of ccKt effective CAI programs (courseware). 
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A hybrid appro^ was employed which fell somewhere beiw^ HO the disciplines of computer sciern^ and educa- 
tionaf psychology. The tmtructionat Research and Ctevetopm^^Ml Department at BYU also contributed greatty \o 
this desi^i iience ac^ro^ wd contributed a i^erspective thiil media hardware must be cteariy a function of 
instructional strategy, and that oid mudt^ls uf GAI, devetopcHl on yesterday's slightly K^opted business or 
scientific computer systems, may not teach effectively. The graduate program in Inslructfonaf Psychology at 
8YU providiid a cadre ot teamed dewtoj^rs, researchers and theorists who ultimately contributed maior theoreim 
of instruction oh which the novel design of iearner-controUetl courseware was based. Most of the'key perswrncl 
from Texas moved to BYU in 1972 to consolidate the cmirseware prpiect^ effectinq a mitosis of the Texas CAI 
Lal>and transplantmg it at BYU as the Institute for Com(:xit^T Uses in Education. 

This paper takes the perspective hetd by memters of this interdiscipftnary ^u|^of instructional psychofo* 
gists, comjxjter scientists, and others w*o have gatf^red to the mountains at Bri^am Young Uni^siiy. Sin<^ its 
establfshnwnt in 1972, the Institute for Computer Uses in Education (ICUEK has Income a cfepartment in the 
new Instructional Research, Oevetopfnent^ and Evaluation Division (IROEK This deviston ts one part of perhaps 
the largest educational technology group assenrbled at mv mator university. A sister Production Slices Division 
provides production facilities in motion picture^ television and any of the conventional sorts of metiid. A third 
sister division incluctes the library wd iemitng n^ouroe centers. Computer^isted instruction (CAt^ is the 
newest addition to BYU's learning r^ources capatwlity. fn addition to ICUE, the tRDE division houses instroc- 
tional evaluation, research and developmait activities not tied to any kind of media. The ttepartment of Instruc- 
tional Development is heaited by Edward Green, The Department of Instructional Evaluation and Testing is 
headed by Dr. Adrian Van Mondfrans. Coltectively« these departments are or^nized under an administration at 
the equivalent level of Ctean by Dr. R, frwin Goodman. These three departments add a research and development 
fcHTus to the more conventional production activities of the educational technology nitrations of BYU- Research 
is enhanced by a ^aduate pt€>^am in instructional psychology, in coof^ation with the College of Education, 
Dr, M. David Merrill heads this departn^nt. 

One result of the rich selection of possible media and methods and the instructional research and theory 
behind their use is that we place our group clearly in that camp which believes that instruction can be approached 
as a ctesfgn science and that the form of the hardware and software, not }ust the courseware, is a result of that 
desi^ science appro^. 

The design science approach to education now evolving at BYU and elsewhere attempts to apply a discipline 
within each of the fcMlowing stages of design j 

1. Analysis of Educational Needs 

2. Specification of Project Goals (based on neecte and valuesf 

3. Oesispi Strategies 

Instructional Systems t^iyi 

Courseware, Hardware, Software 

Organi^attonai Design (Roles for Teachers and Others) 

Evaluation Design 

4. Realization of [^igns in Hardware, Software, Courseware an<i an Organi2atfon of People and Machines 

5. &iDsystem Integration and Testino (for TICCIT, evaluation and revifion of harchvare, software and 
courseware cK^curs at BYU during 1974) 

6. System Integration arKl Testing (for TtCCiT demonstration and evaluation occurs at Phoenix College 
and Alex»indria, Virginia, campus of Northerfi Virginia Comn-.unity College, 1974-1076) 

This paper is organized into three sections according to the first of the siK stages of the design process listed 
above, * 
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Andtvsi& at Etfucationat Needs 



Ftw <u}iit:uiins lujw Mieve the HfeaH/t^ri model of the '"systems appro^h'" which asserts that the prc^ess of 
uinuvatum itesuff* tHjqms with yn analysis of needs in society, natfowing to neetfs or problems in some class of 
eifucdfiunat iiistiiuiiun, then diinvinq^als fiom needs. This rational, Idteal model can only, be ap^oximaled in real 
life, Actuatiy^ 4 pruHHrl hke TICCIT lieqios when ce^ tatr> intiividuais beccmie convinced that ctimputer assisted 
instruction (CAI) has tremendous |>oten|ial to restructure and improve education. These individuals are able to 
fornuitate a plan which \% fogicaliy relate*! to some real needs and are able to obtain resources to evolve and test 
that plan. Extensive research on CAI was conducted at many CAI latoiatories wd centers, in universities, govern- 
mifin; industry iind sctibois. By 1370 this work had established by data and ar^ment that CAI could extremely 
eftectivtr, <*fficu nt and motivating, and that its cost could Income €|Uitecom|:^ltive* The National $ciefice Foun- 
dation's Office of Comptuer l incvations in Educatimi, miided by the advice of exfwrts, invested $tO million, 
stanind m 1971 72, in two mdji.r ^omDnstrattcNns of CAi, One of these, ttie PLATO project at t*ie University of 
lllinf>is usHS a q^ant computer with upwards of 1,(KK) student terminals of completely new design. Their af^roac^ 
IS t() PI uvide tifachers with this resource, leaving it up to the users to develop the instructional desi^s and 
qrgant/atfonaf desi^s. No attempt is made to restructure the rcrfes of teachers in any major way. By contrast, the 
TICCIT project uses state of the-art computer and television tectinology. The hardware ccMisists of two minicom- 
puters and 128 color TV student terminals. The hardware, software and courseware are reflections of a strategy 
for restructuring education toward a learner centered, individualized moctel. Men and machines are organized to 
serve as a source of help for stuitents who are learning and Rowing in a number of dimensions. The concept is that 
computers cannot merely be adcfed as an adjunct to teachers in present educational structures, but that a lar^r 
redesigtn must be undertaken. In the wcrds of Peter Orucker: 

The educators still talk of minor dianges, of adjustments and improvements. Few of them see 
much reason for radical changes. Yet education will in ail likelihood be transformed within 
tlie next (tecades by giant forces from without, 

ft wdl be changed/ first, t^ause it is fteaded strai^t into h major economic crisis. It 
is not that we cannot afford the hi^ costs of education; we cannot afford its low productivity. 
We must <^t results from the tremendous investment we are making. . . 

Teaching is virfiere agriculture was around 1750, when it took some 200 mm on the 
farm to feed one nonfarmer in the town. Wfi|}ave to make the teacher mure productive, have 
to multiply his impact, have to increase greatly the harvest from his or her skill, knowledge, 
deification, and effort, , .<P. F. DnK:ker, 77w Age of Discaniinufty, New York: Harfw & 
f^ow. 19691, 

Given a vision of what can be accomplished and the resources and opportunity to do it, a needs analysis is 
tfie application of discipline in data collection and rational analysis to guicte broad design decisions, rather than 
the process of inscribing focus and direction on a Mank tatdet. 

The data collected had to focus on the following questions: Is higher education the most strategic sector of 
education for the initial mtroduction of a CAI system designed for major positive educational intpact and mass 
dissemination? Within higher ediK:ation are the community colleges the sector which initially can t>€St be served 
by systems like TICCIT? Will they prove more receptive to this type of innovation? Ai^ the costs associated with 
various kinds of instruction tn community colleges high enou^ that certain of therp could be replaced by CAI? 
Whdt courses have the heaviest loa<l of students, and the greatest need for individuahi^ation and keeper levels of 
t*mh*diatu>n7 Wl-iat are the socially and educationally valid motives and aspirations of teachers and other educators 
which must l>e served in a system wherein old roles are chan^d? What are the needs of students which must 1^ 
served l>y a man -machine system involving CAI? 

&)m#? dat»? and analysis l>earmg on these Questions is docur»>ented elsewhere. For this overview, it is sufficient 
to olj^rve that hiyhei education does have high costs, high volume and students who are free to take or leave CAI. 
If is suffering from si?rious financial problems and social pressures which call for rs^pid and dramatic change. Within 
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hj^er cKfuCdtion, cgmmunUY collets are the mast rapidly growing sector . They are dedicated to t<^hin§siufjenf^^ 
and to new forms of communiiv Sdrvicf/not graduate research, jnd are fairly rtfceptue lo iniKJvative ap|Mdc>che$ ta 
uccumplish thmtp qoals. 

Within cominiintty coUeges, his^ructian in Freshmarvand lieveUipmentat mathematics and English. usually * 
accounts fat dl least 25 fi^'tcetit of the total cwnact hours of instruction. A lat fiartton ot this (abuut K> pt»r 
cent I couhllie served tiy two moilular systems of CAf Wessons m math ^d En^ish covering from 12 to 2S credit 
hours ot instruction, depending on Now these lessons are allocated tu course titles, Educdt*QnaJ cestui m co nmun 
ity ^'iiHe|^»s average S3.?6 j:«f student hour. $1.50 in direct tnstructic»Taf costs. Much of this cost coutd potenttaUy 
by replact?<* by a man machme CAI system with greater effectiveness and a potentially lovi^r cost 

<«(.,_•,■■ ... • ' . , . . ■ ' 

Community college srudt^nts share with other students the desire for instruciion whicfi is refevant to their 
neeiis and intt rests and res^tfut of thcrr ttme. They resent phonines'. lack of j»eparation and arbitrary treatment 
on tfie j-urt of tt!achers. Oii ih** utlM?« hand, they value producitive human inreractfons v. ^r- eachers and other 
siudencs am! few would likt^ ^1 collrqu jnsti ncum mediated solely by a machine. Op i eui jHment policies of 
conimunitv cnllencs introduct^ *.itg(? numbets ot 4ucte*^ts who ar6 inadequately piepdred in basic math and EnclisH 
SkiMs ami Thus du^ mueddy to take Freshman It^ v. 't t: >urses in these subjects, or other cuU* jp courses, which depend 
Ofi thfst^ sk4ts. BesidfTS thcM jch*ovemem diffinpft, t^^s "lany Suffer frcrni attitudes of ^.voidar^ce toward study and 
learnmg, M^or study habits and attitudes that vUt v rne bfarro on others for not teadikig them, rather than on 
themselv«*sas tht? M^sponstt^le atit^nts for fTerstuMt ,|,t>Mh, 

The motivfs i>f teachers art? varied. Thi* in tt* r ones are rewarded by seeing intelleeiual and persona^ growth 
in individual studtvtts, by tearnmg and mnim i scholarship themselves and by perfecting their teaching sktlH. 
Those who love then sui)|tH!t matter t>elu. vt* tMr y can ct^tvey som** of the imponder^le vaJues of a IiIh?' at educa 
tion " a reverence to/ k^uyvvti-a^n* vifid it> ak ittxsi exf^ession, an excttemen? in rts pursuit, an aesthetic deliq^rt in the 
elegant, attractive nuances u/ a ;cct, a discipline in moblem solving or scholarship, Tearfiers seeft fair reccwn 
pensti for their fatmrs. They value the* respect ot their students, peers and administrator^ and community. Not all 
teachei s are good teadiers and not ait motives for staying in the te^chuig profession are as laudatory as those just 
mentioned, A man machirw system involving CAI should enhance the stature, influence and opportunities of 
teachers possessed of tHlucationaify constructive motives and attitudes, ft is profvabte that CAf tan c»ily be a 
small influence in this regard, but it should certainlv be desired :o Ijk? a positive, rather than a negative, mfluerue. 

Specification of Proi^t Goals ^ 

Earlier publications cm the TICCIT project emphasized the cost asf^ect of computer^ssisted mstructiou. A 
panel of NSF advisors, familiar with CAI and other fofms of educational technology, had advised that d TiCCf f 
could be no more effective than traditional forms of instruction and could riHiw this instruction at less cost, it 
would be a majcK contribution. A general ^1 of the project was then and strll «s to create a "market siicorss ' 
so that after the application of government funds, private industry will proviffe efficient dtsseminabon and 
competition to iur^er drive the cost down, 

Despite the more limiteri aspiraticms initially held by outside a^ivisors, the desi^ scierrce approach of thr^ 
TICCIT project started with a set <^f educational goals that put cost in its prof^nr context, Iniiially, httle could 
be said about goals which seemed too remote to bp accomplished. As the proper tfvolveii. however, <f*fSign strati^ 
gies became well defmed. These strategies now provide basis for optimism that ^ more ambitious set of goals can 
be achieved. That complete set of goals is discussed twlow 

The Derivation of Goals from Needs and Values 

In c!a',sral stairments of the systematic appfOi«:h to instructional devflopniertt, go^s are derived from the 
nf.tnh of the msfitutidns and the needs of individuals sorvt'd by these institutions. fVeferably thi*se gr^als are 
measuiable at least throuc^ a lonqitudina! research study. It is d^ai to any thinking jM?f son that this modt'l oi rm* 
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systems ap^oach is dfi pver simpMfM:atfcHt. Goals are cteriv^d <^ much front the i^tlosof^y ^nd values of the 
system's Uesignnjrs f com iht? net^ they serre. The r^ds must be there, it is true, or the firoduct of a ctesi^ 
effort will nut in: u^i The goals s^iould indeed be assessable to ^sure they are achieved. 

U IS the exph^tt recognttiotVand use of human goals atid values^ hmvevefp in a rt^ous^ emi^ricaUy bas«?d 
process of ck^stgn amf cfeveto^mtent that pro^ictes tfie critical distirKTlicNn between a desi^ science ar^i anatumf 
sciefKe/Herliert Simon, m his Itttle book of provocative essays on ""The Science of tfw Artiftcwl" rnakes tliii 
distinction. White he does not treat tfie desi^ of instruction directly as me of the artificial sctern^, he deals 
with tearr>fng anif problem sofv«r>g iri this manner. It is dear frcKn his bcKik that he wtHiMcprtsrder educatfCKi as . 
one of those d^sctpUnes which cuMid more prof itaMy be viewed from the perspective of artificial science than 
as a natuf atstTieiice Lwsed field. Vtfe prefer the term '*Ctesign Skrience of Instruction'* to Simon's term* ti would 
be presumptuous to claim that we have i^velc^ied a ^tesi^n science for instr^ticm at this tiiw* but it is appro- 
priate to place the dcs<£piers of the TICCIT courseware witiiin that wiH^td^de group of researchers wfio believe 
that such a science can be itev^loped. ^ 

Tht?re ts a diwrsity in vetoes anumg the desi^ ^oup which generated feamer^ntfolled courseware* 
Religious iMCkgruunds mctude Catholic and f^rotestant, Mc^mon arnl iewi^ wd a variety of degrees of othodQKy4 
Yet there is. a core of Commocr>vaiues reg^ding the nature of man in piHticular^ the roles of stiKtents and teachers. 
Rather than the mechanistic "O*^ (org^ismji of behavior«^ f»ydioiogy, we see students as a^nts. using that mean- 
ing of the term which lekntif les the student as or^ who acts and who has the povM^r to ori^nate i^ion. The 
student's own ^^ats and values shaf^ his choice of ^ions^aiK} any tfescription of his chdic^ behavior must be in- 
complete and exceptive without consideration of tfiese constructs. It Is felt ttiat stuc^ts grow when given respon- 
sibitity for their own choices and that a system M^iich deprives them of choice limits growth. 

Limits to intellectual and personal growth^ for all practical purp<^es, are imposed by ttw teadiin^ of the 
student's culture. His genetic inheritance (^oyides broad limits within v^idi growth may occur, and with different 
cfe^ees of rapidity. Educational institutions are now structured both to unfold human potential along t^tain 
dimensions and to limit it in others. Tn^ting this a^^umpttcm m a vaitiffi^ wtvich instructionirf systems dest^i 

r^tfier than as an hypothesis to be wei^ted by the descriptive methods ad^ted from naturea! science has strwig 
operational conseqimnces. If the limits to ^t)wth are indeed far beyond what can be attained by present educa* 
tional environments, then the designers of new tearniftg envircmments can exp^t c^uite remarkatrfe quantum leaps 
in student growth as better and better dest^s are found- 
Any design fur a new learning environment can include teiK^iers only, or toth teachers and machines. Ex- 
cept m rather limited as|M?cts of a total curriculum, instruction completely by madiine-student interaction must 
be incomplete and protebiy tartly un^rc^ptabte by students. Since the maKnity of a student's life is involved 
in human interactions ami education s^ks to expand his ability to function with o^rs, the process of edt^tton 
as a total concept must involve human interactions as its dominant feature. The con^ter is another technological 
tool to enhance certain as|>ects of this process, those aspects «^ich deal with information transmission and 
processmg and information manai^ment. 

Arteacher can be bf>th hero and yillain, sometin^ both, in the process of bringing about growth in stuctents. 
A teacher can destroy a student's faith in himself and turn him away from learning forever, or break throiigh 
turners that hmit fiis hori/on and tree him to growth. TeixJiers can learn to use computers »Td other tools to 
exfiand their ability to instruct, to mana^, to build students and to produce scholarship. They can also subwrt 
systems which involve ii»f:hno)Dgy and quietly tnit effectively sabotage the potential of such systems. Some pro- 
|)osed roles fur teachers who will use systems like TICCIT are described below. 

While \ drt? considering the values which influence the ctesign of TICCIT, it is sufficient to say that the 
same values tii\dMi to flu* f kocoss and limits of growth apply to teachers as well as stmfents. Teachers are the 
central agents in this prrxrcss, and upon the way they define their roles depends the suixess of the project and the 
hituft! of ti»chn<ift?i|y if) i ^isTniq Hlucattonal systems. 
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Ho iEitten^ hm bew mmto to dniK>tate the sAm^ dt5»:iisskm of values from ^ im^re ttum extmsiw h«^a- 
lure of i^itdsi^y , psv<^o(ogy and theorfi^. No ^xc^hme in these fieUh is ^irnnmi t^r tt^ ctosipiers of the 
TtCCiT courseware, only recognition of the (H»mful role of v^uesln the prrce^ of ctesign. Givw the c^tsiw 
rote of values in the de»^ of systems like TICXIT^ tiavit^ great socki potenttaU it wouki se^ that a responsibility 
exists to makea les^t the itKire influential values exfriicit. X\Hm mentioi^ at^n^ will se^ by the reacfer to 
be implicit in the nniltifde 9^% for sti^lents aiKl the s^is f<^ educatcnrs discussed beiow. 

De^pi Go^ fmr TICCiT Coiimvi^re, Itefdv^ 

TaWe 1 summarizes the dasispi goals. Ir will be seen that they are grouped into the categories of ir^titution, 
content^ individual student and educator. 

TA0LE1 



C€IST OPAL , 

$t .00 per cont^ hour 
25% less time 

ln(^e^ enrolln^t sfgnificamiy 

CONTCINT GOAL 

Smalt step forward in content 
Oarify objectives 
I^i^ for ftexibitity 

GOAI.S FOR STUDENTS 

85% of students will adiieve mastery 
Im:re8s»d effmierK:tes 
Improved str^egtes 

Voiuntary s^^^t}^^, reduced avof d»m 
Responsit^lity 

GOALS ran coucation 

CteHne new rotes in n^na^ment^advisen^t ^ 

Stimulate teachers to ctomonstrate humane values in follow-on or coordinate instruction 

Define new professiof^l roles in itei^ofMnmt 

instructional research— computer as a tool 



tfiffCittttioi^-Leeel Goab 

A, Cost Goa/5; We seek to reduce the cost of a contact hour of community college instruction in mathematics 
or English to son^what less than $1 per contact hour, widi no loss in effectiveness and with reduced contact time. 
By reduced omtact time, mean that it is our to reduce the time it now tak® a student to comi^ete equiva* 
lent work in madiematics or Enf^i^ by some 2S percent or more. 

This cost goal implies that the teacher-student ratio in the TICCIT cours« will be reduced so that com- 
puter-assisted instruction need not represent an add-<Ki cmts. 
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B; €ttn^mmt Go^: It is wir god to iraar^ imrc^timnt si^lftcantly. The c|u»itttstiva cfefinition of 0»e 
word "si^jWcantlv" is to be determined by the Edi^ation^ Twting Service through its summatlw evaluation of 
the TieCIT project. We expect this increase to occur both through reducing attrition in the oHtege as a whole and 
through increasing enrolfment in courses ^ven by computer-assisted iiwruction owr «)y increase which mi^t 
occur .thrpu#rt£3ditional modes. If adiiewd. this factor onild give administrator an option other than ra<kicing 
staff to »^ieve the cost advantages of tlCClT. • 

C. Content Goifs: it was not our goat initiaWy to innovate substantiaiiy in curriculum content. We sought 
»i appropriate and accwate representation of the curricu!Mn!i as now described by professional or^izations in 
the fieltte of mathematics and English, in textbooks and as taught at community csrtfeges- As the proiect has | 
evolved, we have found the math content to comespond doseiy to standard textbook topics. In order to achieve 
the more important effectiveness goals, the Entfish content has had to i^rt more substantially frt»n conventional 
approaches. 

Vfe seek a smaN step forvvard in nwthematics ccmtent throu^i encmir^ng coll^ to com 
intermediate Algebra into one modular system of instructional matertab, and to replace the CoHege Algebra, 
Trigonometry, and Analytic Geometry sequence by the "Math 0 Course" as recommended by the committee on 
the undergraduate programs in mathematics. In the Math O apfwoarfi the student teams the concept of function 
sees it illustrated by the various elen«ntary fimctitHis. 

a larger step forward in EngTish grammar »id comfKisiticKi, F^imarily thoMif^ n^r^ tc^ard a gena* 
tive approach to composition, and throu^Ji ctafifying the objectives and structure of those subjecte around an 
internally consistent, generative rhetoric Our analysis of the objectives of Entf Ish composition and grammar leads 
to an apparent simplification in what "topics'' are cw«idered. Sik*» an analysis does not map as readily into En^ish 
textbook topics as does the mathematics analysis. 

We exfMct to provide great flexibtlitV rttrmitfi a modular structure In both mathenrwti« awf English so ^t 
community college administrators and famlties can utilize these materials in a variety of ways. 

Effecthwnesi Gt»>s f mihfidhid StiHientt 



A. M^tery: At teKt 8B mcent of ^e stwfents v^ take the TICCIT courses will a«*i«eve mastery, « 
defined by the mastery tests at the le«on and unit lewis. 

B. EffKiefwy: Students svill ftnprwe their efficiency in learning from CAI by a substantial factor as mea- 
sured faetw^ the first two and l^t two units of »»y coiffse. iMwv mve m g^xl basis on whic^ to make a 
cpiantitative predkrtion for the nwgnitucte of efficiency iiMrreases. It st^Rids as a ctesi^ go^ norwtfiele«, and is 
expected to contribute to the decreased time of 25 per<»nt or greater disctssed abow, Tinre saving is of value 
to students, as *iei\ as to an educational systcrm, 

' C. /mpmved L&ming Stra&^: Learning strategies are defined operation^ly in terms of patterns of use 
of the learner control comn«nd ian^iage described in another sectiwi below. Improvement in strate^^^ will be 
measured by the extent to which the stiirtent's efficieney imfwoves simultanrously with die reduction in his 
requirement for advice. 

D. Af^fro^^ Rather Tfim AvohimKBs: The students wiH (fevek^ a positi^ attitude of approach ra^er 
than avoidance relative to the sub^ matter in any TICCIT a)urse. Attitude is measured In part by question- 
naires giwn to the student from time to time, but is measured prin»rity by the extent to which the sturfent wilt 
voluntarily work on optional nwterial. This includes AB level materials whid) are not ret^ired, and ^wnes. 
stmulatiOTs, "tidbits" and other itens which are not required. AfH^ro^^ can be nwasured in a SPOss way by the 
extent to which students who take the Algebra course go on and take the hi^r level OHJrse. This a^n^ that 
they would not otherwise have taken it. The same type of me^rement ran take plara in En^ish and can be 
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reftectec? at a arois Itnrei throu^ irtcre^d (mrcrfiment in the more advancM English courses not taught by 
TICCIT, 



E. ft^^cmibUity: Studsnts' attitude of respcmsibiltty tcmard^aming wilt increase from the f ir$t 
unit to the tast unit, VVhife difficult tc me^ifl^e, it is exf^cted that the extent to ^idh stud^ts n^et sdt«duted 
appointments can be assess, as can tl^ extent to v^tch they exert continual eff<Kt toward achieving goals of 
mastery wd effioency. 

The effectiveness goals for stiKtents are very mudi a fuiKtipn of the te;^Riu^<mtroiied courseware c<mcept 
ctescril:^ below. Rather than being ted step-by *step« glided by sonw all-knowing matti^matica} algorithm «i^tch 
^ makes decisions fc^ him, a ftu<tent is given a cormr^HHl l^fig^o^ v^i^ allows him to sun^y freely, ^tdWi^ his 

own sec^jence within t^ a>nstraints set by (H^erequii^tes and estatriish his own teaming t^ics. Lesming t^ics 
are cfesc^itiedtn terms of tl^ seqimnce of ruiQ», examines and pmtice in^wces a stiKimt tt is mly tfmnif^ 
this tearner-controiteii <^rsewdre strategy tttat we hof^ to t>e atrie to help the stucfent c^imre in^m^^n^t (h 
all five of the effectiveness goats discussed abo^. 

60^ for Educatois v 

A iong-range goal of this protect is to make a mo^test rontribatii^ toward the enhivK^iTOnt of the profession 
of education toward greater rigor and dis^ifrfine^ in the application of empirically testeble fHlndples of instructkm 
and man^men t. Inaeas^ productivity and inc^ea^^ prdfe^ionatism is the key, Tfie h«tis for new Nv© 
further ctevetopn^t of a desiyi science foondaticm for education and a related tedhntrfogy of instruMion^ Com* 
(Hiters are a principat tool to administer, mana^, and ctesign instructic^ and to collect data relative to prescriptive 
ttesign theorems. ^\ 

New roles for teachers and modifications of old ones must ef\^ge \^fme these long-range goirts can be 
achieved. Four viMch are of special interest to the TtCCtT proiect aVe the following: 
J \ 
t . Manager-advisors for students involved in TfCCIT courses \?nd other te€^nologicalty nwliated systen«t. 

2. Master teafchers, *; 

3. Designers and developers of courseware, \ 

4« instructidr^ Researchers. \ 

S \ 
Bather than elaborate on these roles in this section, they will be disd^ssed in the section on desi^ strategy 

after context is provtcted by a ckscription of the hardware and coi^seware cfesign. 

\ 

Strategic to Nteet Design Goats \ 

\ 

Since the hardware and software design f<H TICCIT is so heavily influenced by the courseware, whidt In turn 
is ctesigned toward the effectiveness ^ts ctescribed abo\^, the strategy for adii^ving the effectiveness goals will 
be discussed first, followed by a discussic^ of r^w redes for educators. There wii^ follow a ctoscription of the hard- 
ware design to meet the ct^t goah. \ 

Mastery \ 

y 

The stragety to adiie^ t!he goa\ of stuctent mastery is b^^ on the applicatimi of instriKnional theorems to 
the ctesign of a modular courseware data structure. This data structure, or content sfructure, is separated both con- 
raptualiy and fi^ysically in the o^nputer from the logic which implements instructional seqt^ncing strate^€». 
Instructional sequencing strategies are largely left in the hands of the student, who is guided by an advisor program 
to develop his own char^teristics strategy and tactics 
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Tlw instruction^ research, md the iwopostiom or tiirorerm (ferirad frcan it. tM»ieh sh^ied the <tesi^ of 
the conmnt components is described in Memil and &HitMf8li (1973J and Merrill (1973). In the foitner paper, a 
review of th€ Uteratvre on learning and instrt«Jtion !ed to tfie ikveif»»n«it of what is in effect a tsxonmny of 
Instruaiwi^ varlaWes. Bv n^ans of this taxommiy, any imtructionai seqt^ict involved in Comptex cognitive 
teamtrigtasks may be characterized. 

Thiis taxonomy involves three classes of variaWes: presentation form, inter-di^ay relation^tps and 
mathemagenic information. 

Presentation fomi may be of four ^pes, geiw^ities of instaK^s. either of «^Jdi may be pres^ted either 
in expositoty in inquisitory form. The system deals primarily with concept leaning sMi rute using, so a generalitv 
Is a definition of a concept, of a dear statement of a rule. An instance is an examiHe of non^x^Me of a conc^ 
or a rule in use. Expository means to tell. inquisitOfy to ask. Inquisitory genendlties (e.g., "define a conc^**) 
are rarely used in TICCIT, since memorization of rules is not sought and since it is difficult to aialyxe by com- 
puter open-emted definitions or rule statements in naturM ian^i^. 

Mathemagpnic inforn^titm is information w^ich birth (gen-Ho leamii^ (nrad^^ 
invT^wes prompting and cuing and other attention-f<xnsir^ tschnii;^. S^fic t^nit^ indixto attrila«m iso- 
lation (use of cofw. graf^ics. etc., to his^light key attri^ jijes), search strategies (step-by-step algorithm!, 
mnemonfc ai<fe, md prodt^ttion strategies (t^ristics u i»ie ttie ^tx^ic»i of sttxtentterierQisd i^oducts. tor 
exampie^written paraf^phs). , 

Certain concepts of m«vnK8diif» imtructi<m dewekped at dnr Tex» labcnrattry wre combined with the 
Merrill taxonomy to devise the nxxluiw^ courseware stntaMre. Th^ Incduffed the conc^ of hter«rdiicriiy 
indexed data structures and a command livig^sge to mows aiKiut witiiin these structures. A set of OMitent files 
indexed within these structures was defined. &mx the ccmteht files were detffilt^ along the lines of *te taxono- 
my of instructional variables, the idea was i^a^sti dents couid use the command taiguage to sequence these files 
themselves, thus manipulating instructional varbMes. 



The TICCIT courseware is hierarchically oj^|»ii»ed into ftair levels. TIwm le^s are ref^esented to the stutfent 
by special displays that present the hierarchies, lii^t the tofHcs. provicte acce^ to a st^t^ized lenk^i of the objec- 
tives and display status after die student has vwMrk'^. These ate: 

Course Level: Course objectives and status dh^^ (rourse map). 

Unit Level: Unit c^jecttves and status display (unit map). 

Lesson Level: Lesson objective and statm di^tay Ifssson map) 

Segment Level: Primary Instruction Components (rule, exami:^^. practit^}. 

Objective and Status f^piay (Map) 

A simplified rnap is shown in Ft^re 1 , T!^ scrwn displays a hierarchy on one side and toptes m the other. 
To survey, the student may look at the intrcKiuctton (either a mmiie^on, a sequence of digitally generated dis- 
plays, or a videotape). He may also type integers on P. Typing an integer followed by OBJ (objective) key 
grves a cartoon illustrating the segment objective. "P^ gives the prerequisites. STATUS is indicated by coloring 
the boxes red, yeMow or green to indicate trouWe, uncertainty, clear process. Typing "X'' giv« a similar map 
for AS test, AS work, and games, simutatjcMis and other "fun optiww/' 

At the course level, the boxes r^resent unit objectives, at !he unit level, they represent lesson objectives 
and at the lesson level, they represent se^nent ot^tives. 
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JNTRO 




TEST 




EXTRA 




1. The Cbmplete Verb Phr^ 

2. Uses of the Vorb "to t^'" 

3. Metmrnzif^ Be Verbs 

4. Recx>giizif^ Have V«rbs 

5. Recognizff^ the Models 

6. MefiKM'izing Mociats 

7. Icfontifying Verbs 




PREREQUISITES 



Fi^e 1. Lesson MAP 
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To peritiit the student access to any levet of the coursewiire^ a tearner cwitrpJ ccimntand keyboard was 
dffsi^erf. tt is illustrated in Figure 2. Tl^ ATTN key si^is that a typed ccmmiand t& forthcomii^ (e.g., logcm, 
logoff, catcuiate). BACK displays th« iirirm^didtdy (receding screen in«ige« SKIP |»rmit$ t^^iJ^ing a test item 
and certain otfier functions* NOTE records a comment for the authc^- and EXIT pc^ back to a tevei from 
whtdi the stuilent exited for some c^ration. TN? nine keys at the bottom of Figure 2 are iovoi^ in the tearpr 
control of his own learning tactics. The ROLE, EXAMP, PRACT, EASIER, HARDER, HELP, and OBJ keys deal 
with events within a segment while the MAP and ADVICE keys are more generaL 

On a course or unit n^p when the student selects a box, he pushes the "GO" button and dro|H» to the next 
lower map. On a lesson map, when the student selects a segimnt, he pushes the "RULE'*, "EXAMP' (ex^npie) 
or "PRACT" (practice} button to inter^t directly with the content* Fpllcnfving my of these three buttcms, he 
may push EASIER, HARDER, or HELP to vary the instructional variables whidN he may requi^ for effective 
learning. 

FrtHTi the lesson map illustrated in Figure 1 and from the primary iostrtfctic^ k^s may be inferred the 
various content files which compose the modular courseware data structure. 

The three main primary instruction leamer^ntriri buttcms are related to the {Ke^tation form dirmnsion 
of the taxonomy cf instructional v^nriables as indicated in Figure 3. Itiis figure also $hi»ws hmr the E^tER and 
HARDER keys are related to inter-display refatt<»iship variables, and the HELP to mad^emagenic information; 

The function of the nine principal learner control command keys is as foiled: 

RULE Accesses the main generality for a segment. For a concept, this is a definition* for a rule it is 

a clear statement, for memoriaration it describes what is to be memorized* 

EXAMP Accesses the next instant in a file of expository instances. The sequemie of inr tmm is con^ 
structeii so that matching, pairing, and other instntcticmal variai^es, not appropriate for 
student controi, are built in. 

PR ACT Accesses the same instance file as EXAMP, but presents it in inquisitory mode, with necessary 
answer processing.for student-entered constructed responses, 

RULE may be followed by: 

EASIER More concrete form of rule (an andc^). Simfrier terminology. 

HARDER More abstract. Technical notation ^d terminology. 

HELP Mnemonic aids to remember tt^ rule. Attribute isotaticxi of key terms or diaracteristics using 

coior, graphics and audio. The^ displays may be followed by an information prcK:essif^ 
sequence for using the rule or testing instances of the concept. 

PR ACT or EXAMP may be followed by: 

EASIER or HARDER ^ifts to easier orharcfer instances. 

HELP Instance specific attribute fsoiatton usir^ color, arrows, sometimes ^aphtcs and sometimes 

audio* Aids to recall the rule are presented first, follcwved by a step-by-step walk-throu^ of a 
good information prot^sing algorithm for usmg the rule or testing the concept, specific to 
this instance. 
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EASIER and HARDER are tvpicallv 'Mntw display re|atiwiship'" vafiaW^ while HELP fw^ovides "mathams- 
genic information/' aithou^ this dfstincticm does not always hold m the case of E/^IER. 

Matching of examples and ncHi-examples and a default secBience ^neririly ^ng from easy to hard wd 
covering the necessary range of divergency anwig the mstanns is taiilt into th» insti^c» files their contrcrtting 
logic. A principle in tfia dest^ of ieanrm c<witroi was tfiat stuc^ts should be ^ven contrctf cwly th«0 
variables for «vhich they had or could learn a tmis for tnteliii^m:^ dioic^. 

There are five basic kinds of cont^t files wd additic^iai files for display forriHitting and ansvrar-f»^>cessjfi9. 

Map files include the objectives and prereci^tsites for survey, and the INTRO content for cpurse. unit and 
lesson matis. 

Generality files provscte for eacAi sa^nent a main generality, ah easier versi<Mi, a harcter ^sscki attd a "help" 
f ite for the gener altty. 

Instance files incluctei for each segment a sequence of tetween about twelve am* forty instants. The 
instances are clas^if ted » easy, n^iu^n and hard ^d are avaifaMe in expositwy or inq^isitory nK>des. FcM-each 
instance^ a help file specific to that instance is availaUe. In inquisiiory mode« answer^proc^ng ami fee(ft>8ck is 
available. Instance files may be ctefined tiy ^mrative algorithms » v^il ^ by a set of disaete stem 

Test fifes fo*^ each lesson are made up of instamws similar to those found in the inipiisitory instanw files. 
AS level tests are aho avattaUe. Unit and a>ufm level tests may be provided. afthcHi^ tifiey are often off-fine* 

Fun options are games, simulations^ tidbtts of hunxHrous or interesting informati on« and options to Sook at 
extra videotapes of interest. These are macfe available on the same map with the AB work, hi^^fully to InduM 
students votuntarily to choose opticmai work. 

The leamer-contrd command iangua^ provides the student with a means to access any file with few 
restraints. He may be forced to listen to and took at advice if he is going astray, but he is never forced to look at 
any instructional material that he does not select. 

From the above description of courseware content structures, it is ^K^ible to summarize the strategy for 
Sieving the of masteiy . E^ studait has an idic^yno-astic ret^irenw^t for ii^tmction cm the various ob^* 
fives whidi constitute a course. Throuf^ the map displays m6 throu^^ tfm status reports using thh map display, 
he can selm which objectives h;; neec& and, widiin l^oad restraints set by the prerequisite relationships between 
lessons, the secpience of objectives* Within an objective, which typically teadtes a single (incept rule, studmits 
vary <^ the level of abstraction, concreteness, diff icuhy, and the help they need to understand how to perform on 
the practice items whic*i test that objective. The tactical sequent^ po^iNe by various studaits are unlimited. The 
slower students will need nrrore EASIER displays and more HELP. They will probably need a ^igiafer nunrf^er of 
instances. Bri^ter students may use a discovery ap^oach, focusing on the harder practice instances. They will 
have less renuirement for HELP and for the alternate rule displays. Status reports signal the studwit when he has 
achieve mastery, so ^at all students who can read ^e displays are assured that if ^ey keep working they can 
eventually reach a mastery stale. 

Efficiency — The careful aialysis of content into learning hierarchic typically irKreases the efficiency of 
systenryitically desired instruction in comp^ison to classroom instruction, since if^identai material Is deleted. 
Furthermore, students can skip those objectives which they already know. At the level of the tactical seqt^ces of 
primary instruction keys widiin a s^^^ntent, also hope to improve efficacy by helping the student ctevise his 
own characteristic plan of att^k, modifying it as needed. The advisor program and status displays <»nstftute the 
design techniques used to achieve this goal, it is expect^ that learner ^ntrot will less efficient, at first, than 
would a skillfully designed adaptive sequence based on research and i^ntrolled by the computer- The hypothesis 
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IS that given ddet^te status disptays. a rfeiKonaWy good advisor, ami f^^ikx^. stttddfiti can develop skill in strategy 
^ lilies vi^tdt Witt* excml the efficiency po^ 

tiVfmv&f Str^teffl^ - Assurrm that the art m^d ^eni^ of matheiraticai rrKxteting of the (earning |>roce» 
should tfvalye to the point ^ere greater teveh of mastery and effic^ncy could itfwaiys c^tatt^ by amuHitef 
contrd than by learner control (a i^ible future which ifcH>bt will ocxurK Even tlwi, learrw contrcrf would 
he preferred/ The goal of imf^ oved strate^ and its ccmnwiiiKi gcmK impro^ atlituctes of d{H>fo^ mi res- 
ponsibitity^ shcaild not be sutKirdinated to the q^t for efficiency. 

Previous resean^ on learner ointrot at the Unhrersity of T^^^ 
did not seek to estsA;Mtsh reidtii:Hishtps between the avail^Mty of learner contrc^ and the^owth in strategy 
competence, appro^^ responsit»lify. The concei^fcm of teamer controf w» top narrow, tic^ in r^tkm to 
the outcomes of learner control tJ^ tmmn$ to implement it. The availat^ CAI f»nogp^ams wre too slwt in 
duratiun for much skill in leffirner control to det^i^ ami the coi^^^are date structures lacked t»» modularity anu 
the relatfc^ship to instructional variables inherent in the TICCIT cotirsewire de^j^. 

A broader concept of learner control recpiires better answars to die c^tior^: What is to be ccmtrolted? 
How is it to be controlled? ONrt what tmi% do we ex|^ the studtait to team to contrirf It? The taxonomy of 
instructional variaMes cfe&cribc^ above gives a framework for^m^ring the first c^^tion: TNr sttktent shmitd 
have control over imtructional variables vtfhich can make a difference tn his laming. The variatiles rwtew^ and 
classified in the paper by Merrill and Boutwelt (19731 were divic^ into tho» which could readily be manipulated 
by the stucfent and those which« at least for ncw^ slnHjId remain under the c^trcri of tt^ authors arnl tl>e 
computer. 

The results of this ctecision process are descrit^ above in the discussion of the tAfiP to^c, and the primary 
instruction commands. 

How are these variables to be co«ttrolled? Earlier learner contrcrf researdiers hmi refinqui^ied control to the 
student in a fairly ad hoc and ncm-systernattc manner, focause of the 1^ of separation of strategy end c^ntem in 
the various tutorial CAf languages, choice of options was thrust tmeximctedly into the hmnte of students at content 
specific decision points* A more rational approach develoi^ in later y^rs (Schneider, 1972K but these approaches 
were still limited m the rangu of variatdes pij^red under student ccmtrol. The TICCIT desi^i for leame*- contrc^ 
viewed student-machine interaction as a cpmmunicatirKi process requiring a formal ciKnmand language-a lan^> 
age related to the variables which afffict learning, 

A mode: for stuci«>iu-machine conrtmuntcation ctevrioped by Pask (1967) provided one source of in^ration 
for the leaner control command lan^i^ implemented in TICCIT. I^k averted that ail oommunication between 
student and computer can be described as taking place in one or more sf^iai tangua les* The flow of instructional 
information sequenced according to fixed algorithms within the computer, arKi ^e answers to r^estions and 
prd>lems entered by the student comprise what Pask calls the lan^ia^. Discussion abcnit the instructiof^l 
process itself, and attempts by the student to ojntrol the pro^ss in some way, take place in ft is possible also 
to define an language in which control processes can be discu^^ and modifi^. 

In the TICCIT system, we speak of ^o^essivety hi^er lewis of discourse, analogous to Pask's lan^^. 

Level 0 may be implemented primarily within the f il^ of instances wha^e stuttents n^ look at worked 
examples or may practice- Level I is implemented by mi^ns of the MAP logic and the primary instruction keys. 
Level 2 is implemented by an advisor pro-am, which refers to a of student historical data (monitor) and 
communicates by refer^ce to "status displays" at course, unit, lesson smd segment levels. The advisor also com- 
municates through audio and throu^ bli^^olored visual displays. 



96 



The concept of a learner control comimnd t^^ge md MmorwhH^ permits discourse tw^tween stifttent 
unci macbf m at all thtee of Pask's «ev€l« is the key etemem in the dcsi^i approach to improve student strategy. 
The eiemenis missiitg frcmi earlier implem^tations of so<aHed learner control were the tnstrui^icmat variaWe- 
f t l,ited comnwnds, the status d^fays und the ^tdvisur. 

Thest? latfifr elements provkle an amvm to tfte t^ttonr "On what basis may the stuc^t team impro\^ 
^^^^V»<j4es?" Given lostructtonally rdeyatit commands, iiveil-itefiniHl <K>als (objectives and tests on MAP displays^ 
dm! status refHKi s which rewal the discreparN:y iMtwwn fxresent status and desired status, students hw^ the 
mformanon necessary to initiate stratc^c and taaicat decisions. Th<! avaifabiltty of an advisor p^mits the student 
to rfiMt?st ^i^gestiuns on whidi sfrategic or tactical Oecisior® mi^it be ^3|:^ofK(ate at any time durir^ the pr<H:ess 
o? instfuction. The advisor also monitors the sti^ent's choices t iffm unSolicftediKivice about strategy on 
taa a^ whrn the student departs from a generirfly u^f ut mc»iei 

By means tif the advisoi , the goal is to ftelp the student ItMrn to use the status dtspfays to ^icte his cwn 
initf.#tKm of SI rate<j*c and tactical maneuvers, until he becomes ttKfefi^ent of any ret^trement for sdy'mr stw^^t. 

Fiqur*' 4 illustrates the four f^rts of a strategy, along with the commands from the learner control com- 
nxicui langiiaqe ot from the MAP display which allow the stud**itt to control each part of his strategy. 

A sut vey is effected f irimarily through tf^ use of MAPs at the courw, unit ami lesson levels. Any map per- 
m?f > access to the introduction, videotape or mmitesspn, to the objectives^ the prerequisites^ and any rule display, 
Th*« student nwy survey any unit and iet^n in tt^ course freely, tHif he n«y not work on instances or tests nn 
any lusson for which he hiis not complettHl the prerequisite lessf >ns. 

Learning tactics occur vvithin a segment, and use tho primary instruction commands RULE, EXAMP, PRACT, 
EASIER. HARDER^ and HEtP in any seifuence. excefH EASIER, HARDER^ or HELP must always be preceded 
by RULE, EX AMP, or PR ACT 

Of 

Testing tactics rake iilace in ttw practice files for testing, and in the lesson and unit tests. Students get 
three attempts at the iessoivlevei working tests* The st^idonts witli hi^ei aspirations or with grater apr^oach 
resfKimes may also elect on certain lessons to take the XTHA work. Trying XTRA j^ovides another MAP with 
fun ciptions and more advance concepts and rules, Af^ "AB" level test is provided on the XTRA map for which 
only line attempt is permitted* 

Review t«:tfcs are pi^mitted at any time. The stuttent uses the survey and teaming tactics commar^is. Wilhin 
d tf?sson, review mode is identical to initial teaming with the exc^ion that no scoring occurs and the advisor is 
limttet i to a few simple, gem?ral comm*mts about rttviifw strategy. 

Approach vs. Awid^e - VWiat variables effect positive aflect toward leaming a particular content? The 
tajconomy of uariables on which the TfCCIT coursewaic was built is a classification of variable vvhidi effect 
mastery teaming, not af f^. Qie point is clear: It is imposstt^le even to measure approi^h without permitting 
frfN* choice. Voluntary choic*? is a requisite for the meiisurement of an affective objective iLee and Merill, 1972K 

The desigi^rs of TICCIT hypothesise that lea?rter control will also contrilHJte to the dev^opment of 
aiwoach responses. The XTRA menu is one means to implement this coru^pt. The AB level test and extra work 
is flesigned to be strictly voluntary. The extent to vihich students s^nd time on these materials is erne possitile 
way to access the growth of apixoach res|K>nses. 

Effective instruction may be the most powerful variable in prcHiucing approach. A sense of amim^ishment, 
ami a recognition of growing skill at strategy and tactics may lead the student to choose optional work during 
TICCIT instruction ami motf? sig^iiticantly, elect to tak« nu3f4j advanced math or English course rtot using 
TICCIT. which hi* othnrwisr would not have t iki>n. 
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"Hie mtroductcNry vidBo(^3es ami mtnilessom ^ ife^ig^ to f^miucea pcisitiv&attitmfe to^mti tekmg 
each TICCJT lesson. In additic^^ the use of color^ gr^irfua and low-key humn deigned to ligfiten the x&$k of 
learning, 

fteSffOifsttktiiy - tike appro^^ the ^<^h of a swse of re$paro$iility is m mtanm for which tiw con* 
trolHfig vaf iaWes not welt umlentood. Th© mofhitar desi^ of T ICCIT, with its dearly defined outcofT^ of 
mastery and ef fittenqr* provkfes an <H>fKHrtunity to dbis^ve variatkms in ir^lkres related to ra^Kmsibte of 
time and rescHjrCes. The extent to which ap|X)intmenfs are sch^t^ kept proviites a gross ttwstwre. f^y-to- 
day ftuctuattof IS in eff tciewy provide a nwe f it^-grairml m^sure. 

The c^g^ i&trate^ (s bawi on ti^ ^^n^Hkm that ^^owth in resfKHi^bility amm» ^irtien i^pcmsibyity is 
dearly fixed and help is provided to assist the imiivkfciat to cany that r^fsc^silbUitv, An oi^r-ridifig tone f^vacfes 
the courseware and the advisor pros^am- It says weOrdiessly tt^t tt«» respons^tity of the authors H to provkte 
ef hH:ti»^e, interesting instructional resmirces^ mni hetpfut advio^. Tlte re^Kmsibitity of the stuctent is to »lect 
his own goafs (at registration) to plan a sequence of subgc^ and to ^v»y hJms^t actively to the task of achiev- 
ing these ^als. 

The training of proctors and teachers is a key asf^ in the strategy to aditei^ imprc^ responsibitity* An 
iwease in the stuitent's responsibility to auMrai his time and his teaming s^ivttles is ^rc^n^^n^ by a decr^se 
in the teacher's r^^xmsibility. The great lesson of parenthood must (earned by tead»w: To permit tfw growth 
erf your chitcfren or yow students^ it is necessary to let go afKl permit free diqti^. Freedom of choice amm tjiat 
the child or the student can ctioose a course that prodm:es faiiure ^ well as a cour^ which leads to success, ff 
provided with sufficient information about the pr<K:ess tf»at led to sua:esi or failure^ the stt^lent can learn from 
his experience, Teadier s and proctors must team mt to step in apd rw^e a student fr<mi an impemiing error, but 
instead maintain a problem-solving nonjuc^pnentaf attitude and provide help wfwn re^imt^. 

In summary, the desi|^ strategy for effectiveness goals is bised on a review of imtructionat variables effective 
in complex co^itive learning^ particufariy in anient and rule teaming. Certain of th^ varial){es v^e put under 
the control of the student by means of a learner control rannmand tans^iai^. The stud^i his language to 
survey the course, plan an overali sccpience strategy, for learning c^ifecttves, dei^opir^ ^pepf ic tearnii^ tactics fm 
each objective and developing his own testing and review t%tics. Status displays help him focus his efforts and 
make strategic and tcwnfcaf choices. An advisor mogram helps him learn the command language so that his strategic 
and his level of n^stery and efficiency will improve ^muttan^msly. In addition, im{»^<»red attituc^ of appfo^ 
and responsibility are sou^t throuf^ global ^fmrts of the courseware design and throu^^ tiie manner In v^idi 
teachers and proctors are expect^ to interact. 

Roles for EffcHsatim 

The roles of manager-ddvisor, m^er teacher, instn^tion^l designer-developer and instructional reseatcher 
are not new roles. Good i^tl^ teaches at many campuses nt»/v derrmnstrate ail or mmt of tf^ skills discmsol 
below in relation to the exinir^ systems of instruction they deal with, TiCCIT may ouse sharps definitions ami 
distinctions to be made in these roles and will farit^ to a heid issues regarding the distributk^n of a faculty mem- 
ber's time across various roles. The issue of inomtives, both financial and professional, for time spent in new roles 
will become especially crucial. 

The mana^r-advisor role is a sutetantiai departure from that of tieing ^e central ft^re in the dassTTOm. 
Mana^n^ent by objectives. ^ developed by experts in fiekis other than location, uses similar t^lnctpt^. The 
teacher helps the student select goats and i^n ^tions, evaluate the succ^s of those actions, and modify plans 
aceordtn^y. The student grows by assuming responsibility for his own ^>a1s and plarts. The system of materials 
in the cofnputer is a resource structured both to facilitate goal setting^ strategy planning^ ieamir^ tactics and 
evaluation. The teacher is a source of help rathe( than a dispenser of information and a lud^. The teacher and 
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proctor training ccHKses to be ctevefaped prior to implenmitdtion in the colleges are di^^c^l to help d^fir^ the 
new roles of mana^-counseior. A$ noted above^ this rote i$ reiateii to xt^ grcm^ of an attitude of responsibiHty 
in students. ^ 

^ .... 

M^m Tmctm - The TICCIT impiemeniation plan does not tncfu^ explfCiUy the roie.of master t^K:!^, 
for this must occur in more advarn;^ course which fotlmr the baste TICCIT mailt ami EpfHtsI^ series. Teai*»ri 
confronted with the idea of computer-assisted imtruction fear ihat ceruin of the ImjKmderaWe ^ues of educa- 
tion can f^er be conveyed by comtHiters. Most of us have known m outstamiing feacl^ who eicerted a stfiK^ 
influef^HCe on our lives, either as a nKKfel of the kind of person m fK0f^<mai we vrautd like to become, or is 
someone who conveyed a love for a particular sirtsject »k a o^eative applxmrh which may haw enroled our life 
or even changed the direction of our career. Confronteu with a set of cribfectives fw tfie TICCIT ocHirseware, u< 
any systemati<»Hy desijr^ imtruction with n^^^e <^^tves. e^q^rierN»d t^hers immedia^rty fear ^t 
something important rr^iy have i^n left out. When the source of this com^n can cfefined, it often turm out 
to be a matter of style or of professionai jiKlgment in a debatal^ area. &>frattmes it is a con^tex b^iavtor^ like; 
creativity or advanced prc3A>lem solving, whtcli can best beacklnmed in fXHHB ad^n^ courses. Often it cannot 
ckfined or even clearly artkrutated. 

ComiHiters demand the great discipline of nuking things expik:it arKf operaticmaK If certain hard^todefine 
values or goals are not tau^t by the TtCCiT a>ur^, at least now anoi^ati<mal definition of what these ^ings 
are not will exist. Let the master teacher in the more advs^ic^ coui^ den^rmrate in the of wetl-fH^eiared 
students wfjrat these values ^ Tl{^^ thus offers a friendly dtailenge to teacfiers: Take the hopefully wei?* 
trained graduates of the freshman n^th and Er^lish course vyho hw demonstrated expticit op^ational skHts« 
and convey to them the things.whfch the comiiHiter and the teachers in the mai^gier-^fvisor rote could not 
wnvey. 

The proper questions regarding computers in instruction is not "Can computers replace teachers,*' but 
'Tor what subjects and students shouki instructkm carried f:^imarily by m^ine. when orimarily by a teacher, 
and when as a shared re^nsibility between teachers and machines/' By ctemonstrating coufseware and wgsntza- 
ttonal models in two subjects as di^rse as high school through fresfmian a)itege fet^ mathematics and bask: 
colie^ level Ens^ish composition, the TICCIT project seeks answers to these questkms. 

Instructional Desf^wr-Oeveioper - It has teen noted often that a>iiege imtructors are not taught how to 
teach. Yet even the extensive research in te^er education has emphasized Interaction skills, and seldom d^is 
with design and deveic^ment skttls. 

The invention of learner-controlledxiourseware was accompany by the concurrent development of a team 
approach to the des^ and development of materials. These roles are des^ibed in greater detail elsewhere 
{Bunder^n, 1973). Briefly, teachers may serve as subject matter experts on such t^ms. Other prol^kmals 
receive training as instn^ior^l psycholo^sts, who formulate strate^s and who work inter^ively with sutqect 
matter experts in content analysis and content component design. Technkal perswnel serve desi^ technkriarts, 
evaiu^ion technicians, graphii^ specialists and coders or ''packa^rs/' 

One obiective of this pro|ect is to provide documentation and later training, so that others can organize 
teams and develop courseware designed toward their own ^Is and values. • 

The avaitabitity of TICCIT com|;%iter resources for new de^lopment also offers teacher^ a m^ns to 
achieve some of the values not accomplished in the basic TICCIT cou^Si^. The comput^^ can be t^ed fsr r/iodei- 
ing, simulation and the development of small pMkages as adjuncts to the te^^er tnxlassroom arrd iatoratory, 
especially in more advarxred courses. This mode of comput^ use hae been shown to enrich greatly the edu^tional 
experiences of both teachers and students. 8/ n^lng the cost goals arxl thus potentiirfly catalyzing a ir^ss 
market, the'TICCIT project can lead to the v^despread availabittty of low-cost con^ter rescufcvs for teac^er^ 
and students interested in these adjunctive uses. 

101 



trmmwtkm^ Res^ithm- - Re^^ch on tHiff^ toarnif^ ^ instruction as it is (»r^i^ in univer^ties can 
often h»tlv to char^eri;^ os beit^ in^tevant to rm\ tm^i^rtion. Appl^ research, otf)^ hmid, may t» 
w content specific md siti^tion specif k as to pKim^ no s^ner^ity, The des^pi sctetKe af^madi to instructtt^^ 
the modular cwjrs€ware ctata structi^e^ ami *e data reo^ng facility of TKXIT off^- a micfctte ground. 
Comrminity colte^ i^dim* freed of sonw da^^room respcHtsibiMtics, can im^l^te the effects of instruct ionai 
variables, management v^iaMes, and soi^t v^-iaties an of a number of effect! wness criteria, with 
of {^rality. This type of resi^rth tws^ t<K) much of m a|:H>Hed fia\Kir to ^^i^ to most ^fucationai re^^iclms in 
univmities, $t^>^ as they are in descriptive natural K»ent»b^d re^rch paradignos ami (^itos<^i^ The 
f Md fs open to undergraduate iratn^(rs to trake tfie f^ofe!sk>nal contributkms they wiHild like fn rwean*. 
On the or^ hand, TJCCIT provictes tnstnmientation tfl^qwalleied at most educaticmat research c^tors. On the 
mher harei, many college instnictors have research tr^ning, are getting it in connexion with i^^uate work* at 
raarby universities, or can obtain iL ' 

The TiCCtT project as initially concevied made {:N'Oviskm tHsi^tarity onty fw the tr^fisf^ of ^e manager* 
advisor rok to *e faculty of the test cdk^* Hiis «ms to be doite throi^ th« d^ek^mient of a t^t^-proctw 
coi^ to cwiducted at the coll^ps hri?trwifateCy f fritov/ii^ tf»e ^staliatfcm of the e<^H^t^ and prtor to the 
fuH-s€9te (femonstration. As the fKc^ect tm evcrivod, much interaction vwth the facuities of Kioentx Coiii^ and 
Alexandria Campus has revealed the intimate rdationshi(^ mnong all four roles. An implementation plan has been 
formulated and is evolving as of the dat& of this pa{^ to provicte details w h<m alt aspects of physical arrar^<^ 
memVorgantzationai patterns a^d rde definitic^ at each colle^ wilt be tWHlted^No faculty wilt io» their }d)s 
at either raliege* regardless of hoped-for imfmvements inj:»ro€Nicttvity whkili nmy nnrfce asmalier staff feasible in 
the TICCIT courses. Coile^ss can^atiy benefit W turning ^e ^ients of faculty into Imf^ovmimts in more 
advanced course (master-teacher rolet new instructional tievelcfHnent and ii^tmcticHiat r^arch^ but they must 
establish the {»rofessionat and economic incentives and organiiatimal structures to n^ke this possible. 

Nunwmis f mtftutfonat and K^ciotos^cai i^otrfenis must to sol^ tofc^ state iep dators, state boartis, 
trustees^ college administrators and faculty a>mmittees on tenure and f^^ofess tonal development will f^rmit an 
ronm^ in which tto divmte rotes and amtritHftims discussed above can fiouri^. &ich social and institu- 
tional chan^ is beyond the scope of the presait project* As cosigners and ctevek^i^r we can only convey some 
ide^« desi^ the system to provide the opportunities, transfer what information we have and root for the visionary 
and courap<Kis fiKHilty members ami adntnistratc^ at the two coHeg^ to find ways to accon^ish rewatdii^ and 
productive roles for educators. 

For ttose who furtd the colleges^ ihe tctea that an imititution-spectfic research and development capability can 
pay fm Itself in faulty satisfi^ton, stuifent powth and inowsed productivity wiU reiyuire much proof. 

Imtituticmd Goirts 

To adilei^ the low-cost of^ratton f^rojected as a desk^ ^al« the TfCCIT hsardware hmf to to unique. As a 
non-profit systen^ engir^ring a>rporat)on, the MITRE Ck>rfK>ration was able to select f lom existing off-the- 
shelf components* They picked mini^Mf»itm ami tetevbim t»^mok>gy to mediate die TICCiT system. The 
confi^ation OMHists of two NOVA BXiQ mink»>m(Hi^s. a terminai prcKimor ami a mam proce^r« Ttre terminal 
proce^r h^tes comnHins<3tions with 1^ coIck TV terminals. It also pM^ovides keytoard echoing from the 128 
keytmards and formats and refresh^ d^plays (a t^visk>n im^ must to refreshed €0 times p& uicond). The^ 
displays may consist of any cmnbination of diaracters, ref^esenting symbols from any laniMa^* ^^ial characti^ 
fonts n^y to crated "on-the-fly** by a sophisticatid di^lay generator. This same display gerwator is able to 
create distal graphii^ aithw of the n^^matical function variety or of a free-form cartoon variety. Any of th^ 
paphic displays, eittor character or pictorial, may to colored down to a resolutim of % dharximr in any of seven 
brilliant colors. This color dt^lay ami the g^^ic caf^lity is extremely powerful for the fKon^ng and cuing 
a^ec:ts of instruction. 
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T>)e television terminal rrifiy be used .is a di^ay mefiium for videa-iapes, which may to switdiedl from 
a bank of videotape players housed at the central ccMnfHiter, The terminal faroce5«>r handles this OfS^ation. The 
termtnat prpcessc^^ also retrieves the di^ tat gra^^ics from a magnetic disk for display on the television sets and 
retrieves digit^f audio from disks ^nd swaches »t to the terminal. 

The main (K^oc^sor accesses the large data liases of content files (vodiieed by the authors and executes the 
program tcigtc* The operating syst^ is designetf around thv. ^ucatinnal requirements generated by the desi^i 
goals and is more eff krient for this purpose thaft a qeneral purpose time sharing system* 

By using these of f the^sl^f components^^nd mmimi/inii new devei<H)imnt, the MITRE Cc^poratiCKi hm 
been able to keep the cost of the 128 tern inai TICCIT system ctown to a totaf a>st which amortizes to less than 
45 cents per student hour for the hardware. Tt^ cost goat of the entire TICCIT system is to provkle instructkin for 
less than $1 ^ student hour for hardv^re, software, couisewaie and sui^rt personnel* By contr^t a cpnt«n 
hoiit of instruction in a community o^tfe^ ama^ S3.26, This includes alt compon^ts induding r^^icai iHtnt* 
Mbraty and acbninistrat^on. These components can be reduced by tnt^easii^ the library and adminhtration. These 
comfHifxmts can be reduced by iiKrcusing the number of students which a giimn irtrysical pi»nt can supfwjrt; how- 
ever, the matn target is the Sf .50 per student contact houi of this $3.28 total whtdi ref^esents the cost of 
traditional instruction. 

A maior factor in achieving the Cost goat is N>w the faculty r<rfe, faculty ratio aiKl ratio of support personnel 
is eventually defined. Oepemiing on the organirationat ctesign, TICCIT canerKi up costing more or less than 
traditional instruction. Sinc« computer costs will decrease, the major evaluation should focus on ttie effectiveness 
goals, vvhich have received the burden of attetit ion in this paper* 

Desi^ Strategy for Effectiim^^ Goals 

Previous research in CAi has shown that the averac]^ student typkralty f inish^ a block of instructk^n in any- 
where from 15 to ^ percent less time. The probable i^sons for this firiding inducte the afoltity of th« student to 
skip rapidly over that vii4iich he already knows, and nfK)ye at his own pac» otherwise. In addition, careful instmt^tonal 
ani^iysis simplifies the objectives and structure ov^ what is mualiy tat^t in classr<^)m inst rtictic^^ The modular 
structure of the system, the large amount of f reet torn provided by learner c^ntrcrf and the built^n attempt to 
improve learning str ate^es are those designed features of TICCIT courseware whidi should te^ to more r^d student 
progress. 

The goal of increased enrollment is related to the atlitudiriftl cpai of approach vs* avoidance. A positive 
attitiKle towiird fcKith thelTCCIT system anri the subiect matter conveyed by it can influence enrollownt on both 
the TICCIT courses and tho^ courses for which^t^y'df*? pfi»Bi(uisit& . . . . , 
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PLATO^ 



James Parry 
University of Ufinofs 
Urb^a Champaign 

The Computer-based Education Research Latoratcnry of the Unmrsity of iltinois is dei^eto^nf th« PLATO 
IV system to provide high-q^wltty computer b^ed education « fow cost. VWien fuHv imptenernt^, the PtATO 
IV system will consist of a larsp control computer facility simultaneously serving se^rai thousa^ r^otciisers 
of ^aphtcs ter ^Hindis, widi respms^ tkrHW of a fraction of a second. The estimated cost per userfwiil be fifty cents 
per hour^ mciuding c^fierating expenses such as communications ch^ges, managment, and i^riorttzatipn of capital 
costs for the computer and the ^ai^ics terminals. The re^ardi and develofmr^t effort over tte past twehre years 
h^ resulted in important innovations in terminal and cornmunicdtlons hardware and in systernfand teeing 
softwaie* * : 

I . ^ . ...... ^ 

A sketch of the PLATO IV student terminal showing its major mmponents is ^own in figure I. The heart 
of the terminal is the plasma display pmek, now tieing prochii^ by Owens-lllino^ of Toledo, {Ohio. The panel 
n^asures 8.5" X 8u5" X O.S*' and is transparent &12 X S12dot matrix diH'Idy with inherent mermrv« imlividuai 
dots can be v^ttten or erased with(^t disturbing the rest of the drsplay. Terminals ifKorporatii^ plwna panels are 
being built to PLATO salifications by Mag^avox, Fort Wayne, Indiana. These terminals omf ain diarai^r and 
line generators, special keysets^ inpi^/output ports for external ^quipm^t, itnd interfm:® etec^rcxHCS to 1SK> int 
per second telf^f^one tine. The terminal writes IMcharactm per second or draws 60 connected lines per seomd. 
using a data format of sixty 2©-bit words per second (a word typically contains three Gtbit character codes or (me 
IjB-bit panel atMressK The diaracter generator has 126 standard d>aractef^ plus 126 dtars^ers whoie sh^3es ate 
transmitted from the computer md stwed in the terminal's memory. Each diaracter is imi 8 X 16 matrix of dots. 
There is room on the panel for 32 lines of 64 characters. As of June, 1973, over 2(K)^miinats arc cc^nected to 
a Control Data O^rpt^ation Cyber-73 computer. These tprminuJs located acrms ttm country ^nd in Canada, 
with maK>r concentration in Qiampaign-Urbana m6 Chic^o. 

Computer-controlled accessories, also developed at the University of Illinois for use in computer-based 
ediK^tion, are being used widi the existing terminals- A randon^access image selector projects ^y we of 256 full* 
color images onto the back of the transparent pfasma panel (access time less than 0.3 fronds}. The im^tes are in 
a 16 X 16 grid on a 4-inch square sh^t of film. A random-access audio devtce rmrords on and plays back from an 
interchan^able 15" disk made of audio^tape material which holds up to 1^ minutes of recorded ntessages* The 
access time of any of the 4096 starting points is less than 0.3 seconds. Tor^ reproduction quality is sufficient for 
foreign langMage instruction. An input devi« define a 16 X 16 grid of toudi sensitive re^ons at the display sur- 
face through the use of li^t beams. When intersecting li^t beams are interrupted widi a fin^r (or c^her opacity 
c^ject) the position is reported to the computer. Each of the accessories has been dei^lof^d with reliability and 
minimum cost in mind. 

The novel communications technique pr^ntiy in use cuts transmission cmts by synchronous packing of 
Information for one thou^^and terminals into a staitdard television channel (microwaves cxr cable TV). This scheme 
will permit the central computer to handle <Mie thousand terminals a hundred miles frcnn the coqiputer at a 
communications cost of less than S10 terminal per month for 24-hour-a-day dedicated servi^. Inherent 
error rates are tow, and error-sensirwi logic in eath terminal f^rmits the central ccNnputer to corr^ iBrfmost all 
f remission errors. The communications system is diagrammetl in ft^re 2. 
^Acronym for ^ro^ammeti (Logic for Aut<miated Teaching Opefatlons. 
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Hie sysleim softwiire is untrauaf in that no references to mechanicai disk memofii^ are macfe stuffont 
while he studies a les&tm. The lesson and the stiKtem's tndtvtdi^f status reside in ^xU^ided Core Stora^« a 
Controt Data Corporation device v^ose ac^ss time is 5 mtt^oseconds and whose block transfer rate to the 
computer's central memory is WOO million bits f^r seccmd <A fast disk memory has an «:ces5 time of tf^OOO 
microseccmtte* and a trmisfer rate of & mifiic^ bits secorni J Titese untop^ transfer chariKt^istics rm^mtt 
gre«it(y er^anced computer utHi^atton (and lower mmpoter coste) and fraDtional-second r^^xmse times. 
Lessons are compiled in sudh a way that atf students studying a gi\^n lesson ^are a sin<^e copy, ^^ther 
unusual feature is th^, instead of waiting for "end-of-line'% mflery key typed by the student pas^ throi^ the 
coarplete software system before causing some reHK>nse at the tenminai, even if tiiat respond consists merf^y 
of an echo of the tvf^ key on the plasma paneh The result h that the keybc^rd is (smpterety redefinabte by 
the lesson autbor. While the system's primary use is to be direct education, the operatirrg system also permits 
conveniionat data proc^ing at lower priority, including analysis of ^xumutated student resr^jnse data. 

Lessons for on PLATO are written by teact>ers (as opposed to computer p-ogrammersj in the TUTOR 
author language. Se^at thousands of hours of l^son material have fa^n written in TUTOB during the past 
few years. The plasma panel terminal has r^rmitted expand^ grai^ics capabilities, including a>mman(fe for 
prodt«:mg circles, arcs. Sf^tai erase modes, vari^ie^size diaracters written at arbitrary £NigIes, etc. f^owerfut 
ard hj^fy efficient naturaManguage answer-fudging capabilities make it easy for authors to create sophisticated 
dialogs with students. The computational features exceed those of FORTRAN: calculationat statements closely 
mimic natural ai^tna (implied mutttplication, sui^rscripts for exponents, etch and as in older verskms of 
TUTOR all calculations are compiled to madiirw oirie for r^kJ execution, TUTOR is rich in bi inching and 
conditional commands. Some major components of the system softwaife are themselves written in TUTOR* 

Initial classroom use of the system is occuring at the University of Illinois^ and at public elementary 
sdiools and community i^lleges in several cities in IMInois. Sut^ect areas currently being tau#it tndude arithmetic, 
Russian, French, l^tin, chemistry, physics, biology, poiiticai scienw, veterinary medicine, economic^^ i^countancy. 
elementary reding, music, and engineering. PLATO tV is being develt^)^ ^th supf»)rt for the State of iilinots, 
the National Science Foundation, the Control Data Corporation, the Advanced Research F^iects Agency, and 
the Ford Foundation. 
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THE PSI PRCWeCT AT THE UNIVERSITY OF TEXAS AT AUSTIN 



James E. Stice and Susan Hereford 

- —University of Texas 

Austin, Tex^ 

BACKGROUND 

PSt starufs for the *'Ptff sonalimi System of tttstruction" also kmwn as the ^Kelt^ Plafi/' ft w<is oonceived by 
Of. Fred S. Keller, a weH4cfv>wn tearnimi psychologist and was fi«t tried at the University of BrastNa in tS64 by 
Keiter, J. Gilmour Sherman^ Carotine Martuscelii Bori, ai Hf Rodoifo PatL Keiter and S^man tater conlmued their 
deveiupmem of the method at Arizona Stale University^ aruJ their efforts were summari2ed in KeRer's dassic 
|M(K>f , ''Goodbye, Teacher. . 

Briefly, the melfiod requires a teacher to make a carofui analvsis of vwhat his students are to team in his 
course. Having established tfit! terminal and inter*net(iate cft}|ectives^ he then divides the course ir^t^ial into units, 
each contaniinga readtr^assignn^nt, study (|ue$timis» collateral referisrK:^, study probten^ and any neo^sary 
introductory or explcinaiory material. The student studies the units sequentiatty at the rate* t»n» ami (^ace he 
prefers. When he feels hp Sas completelv mastered the material for a given unit, a proctor gives him a "readi nass 
test** to see if he may proceed to the next unit. This proctor is a student who hds been dK^n for his nrtastery of 
the course material. On the readine^ t^t the student must make a grade of 1{X), hut if he misses only a few 
questions, the proctor can probe to see ir the questions are ambiguous and can reword tf^ questions if necessary, 
ff the student does rtot complete the test siicc^fuity, he i« tokt restudy the unit more thoroui^ty and return 
later for anotfier test. He receives a different test form each time he conms to be t^ted. Wo rmitter how many 
times a student is required to retake a readiness test, he is npver penalized; the n^thod requires only that he 
uitimatety demonstrate proficiency. AM students who (itrmonstrate mastery of ail course units receive a grade of A. 

The lecture <s qreatly de-emphdsi?ed i$% o medium for informatics transfer. L^ures may be gn^n at stat^ 
times during the course, Sait only to thosp students who have completed a specified number of units arwl can 
therefore understand the material to tie disaissed. The students who qualify for a l^ure'are not require to attend 
them and the material discusstrd in the lecture rnit covered on any examination. Thus the lecture is used as a 
m^nf. The professor gives his lecture on a topic he is f^ersonally interest^ in and about which he feels he has 
something worthwhile to say. Those who attend corn** tn^cause they wBnt to. 

In various articles and presented papers, Keller has listeci the following five essential features of the PSI 
method: 

1. The go at yoiir-own-FWce feature, which permits a student to move through the course at a speed 
commensurate with his abihty and other demands uf>on his time. 

2- The unit-perfection requirement for advance, which lets the student go ahead to new material only after 
dt»monstrating mastery of that which preceded. 

^ ICdll«r, fred 5. . "Enn^nOTrin^ Peru^nMi/sii Instruction in the CiBi^raom," Revifta Intdrsmcf fcsna (i« P%icoiag^fi, 
^'OL 1. NO. 3, 189 197 (19671. 

^KciiBr, FfPfl S., ^'Gooiibye^ T#?.irh€»r The Journul of Aw>l»eti Behdvior Anaiytit, Vol. 1. No, 1, 79-89 

iSvriryg 1968^ 
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3, The use of lectures ds whides of motivation, rather than as smirch of critical infonmtkin. 

4, The r^ted stress upon the written mwd in teadter-student communication; end finaiiy: 

5, The use of f^octors, which (imnits rej^ated testing, immediate scoring, idmost unavoidi^te tutoring arHl 
a marked ei^iHicemem of the personaUociat ssp^t of the educationat i^rocess* 

PSIatttw University of Texas at Austki 

At the University of Texas at Austin, PSl was first tried in the Fall of 19^ by Carl F. Heteford, Luii F* 
NataUdo and Walter Stemting. They us€d it to teadi Fsyc^tog^cal Foundaticms of SeooiKi^ Edtmation and 
^tliough their course c^ite successful famf h^ t:^n taui^t as a mcKiified PSf course e^ sitmh they made no 
particuidir effort to ^ivertise what they were doing. A$ a result «ie camfH^ at targe w^ i^rotant of their experin 
ment Thus it was not untH ^at PSI w^ applied tn am>ther come at Texas. In the Fait of 1969 tk. BiUy 
V. Koen, Assistant l^of^sor of Medianicat Er^tneerir^, u^ f^l to teach tntordiH^tion to NuOear Reactor Theory, 
a sanior-ievei elective course. His efforts w»e nmt with a cmain anKaiiU of adminMr^ive resistam:o, but Koen 
persuadted his cfepartment chairn>an to give him sorm extra secretariirt a^tstance ami son^ fuiKis to pay for the 
reproduction of units and readiness tests. His course was decidedly succ^ful and to cm knowted^, it was the 
first rei course in engineering taught anywhere. 

Koen a report on his course at a Teaching Effectivene«j Lundteon sp«Mtsored by the C6ile^ of 
Engineering in the Spring of 1970, and in the Summ^ of 1970 Or. Lawrence L Hdt^ock of the Mediantcal 
Engineerirm Department used ?Si to teach Kinematics ami Dynamics of Mediantcal Systen^. His course v/m 
also sufxessful and in the Fal< of 1970, two vmtB en^neering i^ofassors instituted PSI courses. Or. Oiari^ H. 
Roth, Jr. of Electrical Engineering taus^t Digital Systems Engineering t and 1^. Gerafd Wae^r taught a 
gr«luate course in Applied Statistics. 

There now existed a nucleus of four engineerii^ professors on the Texas campus who had ejq^erience with 
the PSI method. AH had obtained results which were superior to th<^ they were accustom^ to Sieving with 
conventional instructional techniques md ^y were enthusiastic about PSt. They bei^ to prc^tytiaea and gave 
unselfishly of their time to acqiaint others with the method, tn Spring 1971 Or. Leonard M. Simmon. Jr., of 
the Physics Departmem u^ PSI to teach Electricity ami Magnetism, In Skimn^ 1971 Mr. ^tohn J. Kn^^htty and 
Mr< <tohn L. Sayre of the Graduate School of Library ^ience tai^t a ^1 course in Basic Beferenm Sources. In 
FaU 1971 Or< John M. White tau^t Prmdples of Pmnistry, tk. PbxA E. Nacozy of Aerosf»ce Ei^neerii^ tau^t 
Er^'neering Con^iutation and {>. Rc^^ert 0. Brooks of Radh^TV-^Fifm tatq^t At^o Envtnmn^nts using 1^1. 
Ail these course developers sought and obtained the advice and ooumel of the ortginai four "expats" from the 
College of Engine^ing and ail obtair^ ysn^ results and are still using the K^ler Plan to tea<^ these courses, 
tn succeedir^ semesters the nun^r of PSI oour^ tm cominiffld to incr^se, until at the f»^nt tims we can 
identify thirty courses which are taught in IS itefmitments in six colleges (for a chronological list of PSI 
courses develof^, see Appendix), 

Tfie Atfied P. Soan Foftmiation Project 

Continuirn development of Keller Plan courses b^n to run into sna^. Classroom requiren^nts arK< cost of 
reprtKluction of materials, proctor salaries and unusual grade distributions began to focus miministrative attention 
on our ^ivtties. This attention was not nec^ssariiy hostile, tnjt we v^re (^coming f^-etty visible ami d^rtment 
chairmen and deans had rr^^ ^ requests funds arxj bud^ts to balanra. They began asking a numtor of valid 
questions {which had p . ousiy already occurred to us). We also wi^ed to see the developn^nt of new courses 
continue in an orderly fashitm and wished to be able to achieve some o^ntrol over their quality. Thi^ we began to 
explore the possibility of attracting grant supf^rt to aid our project. Our early efforts were unsuc^ssful, but the 
Alfred P. Sloan foundation of New York City becan^ intei^ed and they ultimately funded a tiro-yeer program 
which had two purposes: the deveiopn^nt (or redesign) of twelve PSI courses ami the evaluatit^ of course data 
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The courses to b<? cf9>relo|>ed umfer the grant are: 



1. fntroduction to Nuclear Reactor Theory - 8, V, fCoen (redesign) 

2. Dynamic Systems Synthesis - L. L. Hol>erock (redesign) 

a Ef^neering Statics P. E. Naco/y and W- T. Fowler (red^ign) 

4. (¥tndpl^ of Ai^io and Visual Production - R, O. Brooks (redesign) 

5, PrincipH of Cherr.istry - J. M. White (redesign) 
& Structural Dynamics - ft. R. Craig 

7. Design of Structural Systems m Timber - O, W, Fowler 

8. Process Analysis ar«f Simulation - D. Himnneibfdu 

9. Etectr^Ccil Engineering LalKjratory ! C, H. Roth, Jr. 

10. Electronic Circuits f - W. M. Clark 

1 1. Baste Library C^taioguing and Classification - ft^rs. Siiiie G« Herring 

12. tmroduction to Eitgineering Analysis fPre-CalculU!* WathJ - R. Wagiter ar^i Mrs. Nancy Hamirton 
The questions for which answers ar»? I>eing sought: 

a. Do students fcarn more (or bcttt^i ) with PSP 

b. Are the higher average ^ades in PSI courses {ustified? 

c. Oo« a PSI course result hi a signthcdotiy differait long-term retention of course content by students? 
d« Does PSI cause students to f&arn how to study? 

e. Are PSI materials transfer rabfe? 

f. VWiat are the costs of PSI courses^ 

g. How can your course be designed to niintmi^e procia^tination? 

h. How do we reduce the percentaq^* of drornKits? 

i. What is the effect of dass size? 

In addition to course development and study of the questions fH>sed« the project induttes a PSf Summ^ 
trtstftute in Lngineering statics for community cnl^e teachers. Engineering statics is a core course for alt engineer- 
ing students and it is normally taken during the second sem^ter or the s^rni year. Few rortimunity colie^ offer 
courses m engineering statics, either because the enrollment would too small, or because no faojtty men^ber is 
qualified to teach it {there are few engineers on community coMe^ faculties). As a result, the community a>tl^ 
transfer may at a decided drsadvanta^ when he transfers to a four-year college or universrty to pursue an 
engineering degree, 

in an effort to alleviate this problem, Wallace Fowler and Paul Narozy devek>f^d a PSI statics course in 
Spring 1973, In Summer 1973, they revised fhf cour^ and taught it to 24 a>mmimity collet teachers from 16 
states* The material were further revi^ ami have been distributed to the surmner schMt partidFsnts, who are 
using them to teach the course at tl^ir respective campuses. l>)e participdnts at tfie Summer Ir^titute were an 
exceptional group, capable, conscientious, drnl enthusiastic* This part of the Soan Foundation project will yield 
informatics about Question e {are PSI matet taK transferrable?). It also disseminata use-tested PSI materials so that 
the users are not required to develop their own courses. They in turn will be able to sug^t further modificatior^, 
based upon their uxi^rif nces m the hvM\, 
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TNe evBtuation phase of the Sloan FoumfalfOf) fH^oject is umler the dircrtion of Dr. Susan WL Hereford, 
assisted by staff menHiers of the University of Measuremem ami Evaiuation Center. 

Tr^s in l%)*Rdated Ri^eardi 

The conscientious introduction of any innovaliw apf^o^h to teaching necessitates that the imtructor 
diannef increased amounts of his time into preparation for the cmirs^. Hie preceding de»H:ription of the basic 
of^ations of f^t courses should make it apparent that instructors chcbsing to design and implement KeUe^ 
Plan course typically fimf thennelves with little time to devote to thc^ough and systematic evaluation of the 
ef1ecttven<^ of the method or to rese^ch directed towani its analysis. FurthernKire, it is (cTtcwn that 
instructors directly involved in prepartmr instructional materials or designing new educational environments 
frequently are unable to analyze them objectively tecause of their level of bdief and personal involvement* 
These considerations, con^timl with the relative r^ncy of Keller's publtsht^ description of the method* 
have limited research and evaluation activittes directed toward systematic analysis of the instruction^ method 
tts^f . Most published papers regarding f^t are descriptive of applications of the method to specific courses. 
Grade distributior^» rates of student progress, unit structure of the course and stiKtents"^ reactior^ are usually 
re{K9ited. 

Other literature describes the effects of various modifications of the meth<Ki4 The impact of such modifica- 
tions as the removal of seif*pacing and the addition of interviews or oral exami natic^s have been reported. 
Raiearch comparing relative student adtievement in PSI and nuire traditionally tau#it elites also hi^ b^n 
reported. The general conclusions to be dravyn from published reports of attempts to apply Keller's system of 
instruction and modifications of ir are similar to those originally {presented by Keller. Students apf^ar to learn the 
content of the course more effectively and leave tl^ <^rse having more ^itive flings about it than is typically 
the case in classes taught by various lecture-discussion methc^is. 

In his 1967 paper (1 h KeMer sus^ted that systwurtic researdi regarding elements of the personalized system 
of instruction could fc^in only after initial prc^tems involved in instituting the appro^ had i^n solved: 

The work done thus far to det^lop this kind of 
individualized teaching within the mass-eduuition 
framework of a lar^ university has been aimed 
mainly at establishing a base-line procure. Once 
this is dor^, it will imssible to t^n ^sessing 
the relative importance of some of the variables 
involved, (p, 195) ' 

New Directions in the Evaluation of PSI 

Keller's idea of "assessing the relative imiK>rtanCT of some of the variables involved" imj^ies an internal 
approach to research regarding PSI. The evaluative ap{»-oach'in which PSI courses are compared with more 
traditionally tai^ht courses in terms of student achiewment arul effective res^nse serves to establish and ensure 
the credibility of the method and for this red»}n« should not be abamioned* Ttm internal ap{»t>ach to evaiuation 
requires an analysis of the elements of PSI and the interaction between those elements and factors fxesent in 
the general academic environment. Critical examination of some of the possible objections to the use of the 
method is also necessary, ^ ts an objective assessment of some of the daims m^ie in its favor. 

Research and evaluation activities are proceeding concurrently with (tevelopmem and implementation of 
Keller Plan courses at The University of Tex^ at Austin, with the mi^jor source of support at this time being the 
$340,(K}0 grant from the Alfred P. Sloan Foundation. Research efforts by the evaluation staff focus on individual 
courses as well as on the project es a whole. At the level of the iridividual proj^, evaluation activities are designed 
primarily to provide the instructor with systematic feedback from his students and his proctors regarding his 
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devetpping cour^* More candid stiideni nrsponsc^ are typkralty obl^imd when data <if& toHected by p^otis not 
dfmrtty associated with the course or the jnstf uctor. (%vk>u$iv, data from each courje wiit tmxjme part of th^ 
fi'iaf analysis of the total i^^ofect; howe^^ in order to go lieyoiwi typical PSI evaluation efforts of the past, the 
t«^esent study has been designed lo cut a^c^s course, departmenlat, atul cnile^ boundaries, thus focusing on the 
rr^thud itsetf amf the crucial vari^tes interacting with its af^icalioa Of nere^tty, ite evaruation effort includes 
th6 search fw new approaches to educatlonai rese^h since Kelter Plan courses by ctefinttion rute out many of the 
assumptions of traditionat educationaf research strategies. One example of this prohfem is the consideration that 
an instructionat approach which is d^i<r^ so as to imtividudi/e instruction cannot be analysed com f^etely by 
nwans of conventional group m^hmfe sudN as o^mparisor^ of class avera^. The use of criterion based testing 
t»^o(^dures rather than conventional normative testir^ further complicates the evaluation. 

f^esearch and evaluattan work rau^rdifHi f^f wwes ^'t The University of Texa? at Austin can be con^^ptuaiired 
in terms of sev^^al mafcHr areas of a>nc€Nrn yy^ich ^uld be constcteft^ as supfrfemwnts to rather than sut;si«tuies for 
more trac^tional fom« of evaluative comparisons which ar** r*oceeding o^nct^rentiy* 

^udmt Characteristicsi 

One of the primary concerns of the PSI evaluaticu^ protect is the relationship between student charactef istfcs 
ami the relative effectlveitess of PS! courses. Experience has msrie it clear that tmain stidents me simply un^ie 
to function adequately in a P$l learning environmwt, A cond^inatidn of exper i€n« and intuition tells us that the 
aiticat factors here are profa^y not intelfectual varices. Giv^n mtequate level of tnteflc^ual functioning to 
copewith college level teaming exp^ienoes, an individuai's response to f^l si^im to be related to such attiludmal 
and personality factors 2» autonomy, need for interpersonal competition, f^ed for affiliation and motivation to 
earn hi^ grades. Various aspects of a student's badcgroumi and oirrent life style may also inflimnce his response 
to Keller l^n courses. We are currently investlg^ii^ the reiationi^ip between student re^nse to Keller Plan 
courses and back^ourai variables, concurrent situattonai variairf^ such as marital status, emfMoyment, living 
arrangentents and preferred pixe of study, and attitudinal varices such as gei^ra! acceptance of education. We 
are also attempting to detemtine whether stifftents who exi^ess initial reservations about taking a oourse taught by 
the Keller Plan do in fact perform more poorly than they would be expe^ed to perform as a function of their 
academic aptitude and grades in other subjects* 

The major go^\ of this study of the relationships t^twccn student cfi^acteristics and instructional outcomes 
under the Keller Plan is to obtain data whid> may assist in futtire differential placement of sttKfents in courses 
whii^ represent optimal leaning sitt£9tions for them. As ^e number and v^ety of available instructional strategies 
increase, it seen^ appropriate and r^cessary to attemi:^ to find relationships which will allow students to be 
^sisted in choc^ng learning environments which will maximize the probability of their succ^s. 

The impact of tfie proctoring experience on students fitting that role is also uni^r investigation along with 
an analysis of the characteristics of students who become sut^sful proctors. It is (Kedicted.that the proctortng 
experience will be found to have impact beyond the reinforcement of knowledge of the course content. For ^xampte, 
proctoring may in some instances stimulate a ctevetoping interest in teaching as a profession. 

Cost ami Time FMtors 

Ote corsistentiy mentioned source of concern with regard to Keller Plan courses has t>een that they may tie 
prohibitively expensive, particularly in the area of instructor time. In response to this concern, we are dttemptiny 
to tr^ cost arxj time patterm throu^ two or three offerings of a r3i course. It has been predicitfd that cost and 
time will be relatively high during the initial offerirq of a P$l course but will drop in subsequent semesters to a 
le^ a>m|:^rdb>e to that for more traditional forms of instruction. A related consideration is dass "ize. If it ts 
feund that targe classes can be taught suo^ssfutiy by one instructor and a ^oup of proctors and assistants, *t is 
conceivable that Keller Plan instruction tmy eventually represem a reduction in the rest of instruction. 
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RSI WftMn the Comamtomt A»f«mic Settif^ 



M«ny qu^ifora *0iit the fe^biHty of Ketter Plan imtniction haw tw^en oot^rr^Ki with the mtem^ation- 
^ips of f^l courses and the remainder of the curricdluni. G^iestion^ of semat tyi^ have raked. Given the 
hig^ level of reqi ired proficiency, how inany PSf oot^rscs could a stmlent le^CMiably expected to take during 
one Dniester? CoukI an entire curriculum be built maund Keller Plan trttruction? If a stud^t is takii^ sc^^f 
traditronai couries and one PSi ccHirse, mil he tend ;o ne^ect the 1^ course in fa^ of othe»r OHirses having 
schedules arwl precise time requirement . Can complete seifi^iN^ng be permitted within the constraints of the 
«ernester system? Such r«Mt;»t;uns hawe im^fcation« frir student placement as w€^l ^ curriculwi design; th^fore« 
their ransideration is cruaat to fuim e imjprfenwntation of Keller Plan cotffs^. 

One of the usual re: jlts of placing a PSf wi^rse within a conventional curriculum is a relatively high drop^sut 
rate in the PSt course. A p^iod of procrastination usually f^eced^ dnnH^ng wt, A major ra^rd) and evaluation 
effort currently is being directed toward anaiy zing underlying reasons for prcwastinatton ei^ttiat drc^HHit. 
Strategies for modifying in such a ^ to decrease proci^inatlon and the nmjiting dr^H^ rate are also being 
exi^reii Care must be takf^n in this area. hc»A^v^. siru^ strat^^ ^nd to be succ^fuf in reducing procrastitta- 
lion and lowering the drop-out rate rruiy also intrcKtuce elements sufficiently avmf^ to violate the ration^e on 
which PSt IS based. * 

Tr^isf^^lity 

Wh >ther a PSi course dcveibped ami refined at oi^ imtituticm can be transported to another institution and 
used effectively has been another souroe of concern rs^rding Keller Man courses. We ^e pr^ntly investigating 
the effectiveness of the su .nmer institute whidi was descrfiimi previously* Factors under investi^tit>n indude 
insuuctor ci^racteristics, student diaracteristics arrd institutional dtaracte^Mics* Tim feasibility of transfswii^ 
PSI materials i3«tvtfeen institutions has a direct retaticm^ip to the isst^ of cost dful time which was mentione'i 
previously. If transfer can km successfuiiy act^miplii^Ked, much duplication of instructed effort could be 
eltminated. , 

Generd Impact of PSf on S^uitents 

* 

Claims, usually t^sed on theory or intuition, have frequently been made that the experierKe of taking a 
PSI cour.e has a laiti^^j lA^pact on <t :«»udent's study ««auits afKi orientation ^ well m on his retention of course 
n^teriai atnl subsequent academtc perfiKmai^. Tlie indent r^iN^ch ai^ ev^uatlon fNO^ includes an attempt 
to ai;$es$ changes in study hdbit5 araJ cn^iernation ami to monitor stuttents' performance in later courses, particularly 
those for which the PSI course furrctions as a prerequisite. 

f natructifm^ Obfectiim and PS! Cffeethmw 

It s well known that innructionai strategies must be desigiied in sudi a way ^ to be congruent with the 
educat k ai objective of the course. The types of instructional ob^ives (i.e., co^itive, affecti^ and/or 
psychomotor) to which PSI is best suited are airier ar^lysis. Results to date indicate that cA^ctiv^ in the 
cognitive and psychomotor domains are most consistent with PSI strategies* It is dear that the 100% mastery 
criterion would be diff i^^ult to apply in areas where cc^rect responses to re^iness test questions may vary ss a 
function of theoretical orient^ion. 

Eienwntt of PSI 

It was memioned previcHisly tf^t comparisons of PSI and n^ne conventiort^ fonm of instructicMi typically 
have indicated higher levels of student achievement in PSI coun^, as v^l as more fKisttive studam attitudes. 
Althornh son>€what limited at this time, because of the newness of the proiect, our data indicate simitar tremis. 
it cannot be determined ^rom sudi data, ^^^wever, just v^at element or elemems of PSi cause such improved 
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p^rlormanoj. If, for e^ampte, s^f -pacing and the use of i^actors were removed, would the simile tntroduction of 
a oriterion testing procedure with a high required level of pr ofid^cv yield the same t^uiu in the area of stuctent 
achievement? Woutd the a^iditton of imnrtors to a conventiomi tei»:hfng environment yield attittKiinai {m{m>ve* 
ments comparable to those found when alt PSI elements are aw^ied? C^tafnii^ data nece^y fm ar^lyxlng the 
relative ointrtbution of PSI eten^nt^ is diffii^lt since it necessita^ eK^imentai control farely pmsible in 
educational evaluation settir^. Analysis of the relative contributions of the major elements of PSI to its general 
success seems to have the neatest imemial value In terms of researdi regarding tt^ instructional method as a 
whole* Such informatton^ combined with that regarding student cfwacteristics and the tnter^ion tie^^mi 
eduGtCional c^jectiv^ arKi teaching methods, would encourage the esta^islmient of em{»rii»l and logical criteria 
for clKwsing an^ng teaching methods arsl pfwing stiri^ts in da^ deigned to fadlitate their learning. 

Sumnmr f^i Sympmia 

In order to disseminate the results of the project, tvro Simimer Symposia wUI be held. These will occur 
during the S^mmw of T974 and 1976 and ;rfans are to invite ^proximately thirty selected ^ucetors to each, 
paying their expert. These symposia will last tm> to three cteys 

Advisory Board 

An Advisory Board has b^n afHKsinted tot this pro^, both to provide guidance to project pmcmnel ami 
to serve as annther avenue far dissemination of nasutts* The &>ard con«ists of six permanent fy^n4>ers and 
varying numbers of temporary memt^rs who serve as consultants in particular specialty areas. 

The permanent members of the Advisory Board are: 

Dr. I^vid G. Bom, Professor of Psychc^ogy, Ur»\^ty of Utah, Salt Lake City, Utah 

Or. Ben A. Green, Jr., Center for Per^Hiatized tnstruaion, ^orgetowm Uniwrsity, W^ngton, O.C. 

Dr. Fred S, Keller, currently Visltii^ fVofessor of Psychology, Tex^ Oristian Unhfersity, 
Worth, Tex^ (for^ -^ly with Western MkMgan University) 

Dr. David T. Pratt, Assinriate Professor of Mechanical Engineering, Washington State Univmity, 
Pullman, Washington , 

Dr. J. Gilmour Sherman, Professor and Chairman of Psydrotogy, Georgetown University, 
Washington, D, C, 

Dr. ^n G. Truxal, Dean of Er^irreering, State Untver^ty of New York at Stony Brook 

Consultants who have served the project so far arc: 

Mrs- Amo^ne F, I>Vaney, Professor f f Engineering and Mathematics, An^rillo Colle^, Amarillo 
Texas 

Or. Gerhard F. Paskusz, Profe^or of Electrical Engjn^ing arxJ Assodate Dean of Engin«ring, 
University of Hoimon, Houston, Texas 

Dr. Bernet S. Swanson, Professor of Chemical Engineering, Illinois institute of Tedtnology, 
Chicago, liK 



115 



PSi COURSES TAUGHT AT 
THE UNIVERSITY OF TEXAS AT AUSTIN 



FALL 1966 

Educational Psychotogy 332S - Psychologicat Foundations of Secomldry Education - Carl F, Hereford, Lui^ 
F. S. Natalicio. and W. Penning. This course hiK been taught each semester since* It is ikjw tat^ht 
exclusively by PSI in sewn or eigm SO-student sections at a tinn?. Cumact £k. Hereford. Fiuit^or of 
Educational Psychology. 3^5 &inon Hall. 

FALL 19» 

Mechanical Engineering 361 E - Introduction to Nudear Reactor Theory Dr. Silly V. Koen, AsscMriate 
Profe^or of M^^l^nicat Engineering^ 418C Taylor Hall. 

SUMMER 1970 

Mechanical Engineering 324 - Kinematics and Dynamics of Mechanical Systems — Or. Lawrence L. Hnbnrock, 
Assistant Prof«M50r of Mechanical Engitiwr ing, 41K; Tsylor Halt. 

FALL 1970 

1. Electrical Engine^ing 3^L - Digital Systems Engir^ring I — Dr. Charles H. Roth, Jr., Prof^or of 
Eieciricai Engineering, 510 Engin^ing Science Gkiiiding. 

2. Mechanical Er^neerii^ 361 E — intioduc^ion to Nuctear Reacted' Theory {repeat) - Dr. Billy V« Koen. 

3. M^hanical Engirreerir^ 3^R.1 ~ Api^ied Statistics - Dr. Gerald R. Wayner, Assistant Professor of 
Med^rrical Engineerirtg, ^16 Engineering Ldtmratories Building, 

STRING 1971 

1. Electrical Engineering J&iL - Digital Systems cn^neering I — (repeat) - Charles H. Roth, Jr. and H. Kerner 

2. Physics 416 — Electricity and Magnetism - Or. Leonard StmoK^ns, Asst. Profe^or of Physics (Now 
at the University of New Hamf^ire) 

SUMMER 1971 

1. Electrical Engineering ^OL ~ Ditital Systems Engineering ! {lepeat) — Charts H. nuih, Jr. 

2. Library Scten<» 340 — Basic Reference Source - Mr. John J. Knightly ami Mr. J<^n L. Sayre. Mr. Knightly 
is in the Graduate School of Library Science at UT/Austin and Or, Sayre is Librarian, Philips University, 
Enid, Oklahoma 73701. 

FALL 1971 

1. Aerospace Engineering 211 - Aerospace Engineerif^ Computation - Dr. Paul E. Nacozy (Assistant Professor 
of Aerospace Engineering, 227 Taylor Hall). 
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2. Chemislrv 102 - Principles of Ctwrniary - Or. John M. White, As^ate Prof^sor of Chemistry, 1 19B 
Chemistry Buitding. 

3. Electncdf Engtmtring 3^L Dtyital Systems Engineering I (repeat) - Or. C. V, Ramarmirrthy ami Terry 
A. Welch (Using Roth's materials), 

4. Library Science 340 - Basic Reference Sources (repeat) - Jutie Kchteter and »k>hn R, Wheat (using Knightly* 
Sayre maferiats) 

5. Mechanical Engineering 390R.t Applied Statistics (repeat) - Gerald R. Wagner. 

B, Radio-TV-FHm 341 - Audio Environments - Dr, Revert D. Brooks* AsMciate professor of Radio-TV-Fynfi. 
210B Social Work Building. 

SPRING 1972 

1. Aerospace Engineering 21 1 - Aerospace Enginwring Computation (n^at) — Paul £. Alacozy. 

2. Aerospace Engineering 367K - Flight Mechanics - Dr. Wallace T. Fowler, Associate Pnjfessw of Aero^:mce 
Engineering, 227 Taylor Hall- 
s' Architectural Engineering 222 Architectural Engim^tng (Computation (rep^^-^t) Kenneth Armctmng 

(using Nac^zy's materials), 

4. Electrical Engineering 3S0L - Digical Systems EngJnwring I (rep^t) — Ct^rles H. Roth, Jr. 

5, library Science 340 - Basic Reference Sources (rc^t) - •kilie Bich^er ami ^n R. Wheat (usir^ Kni^tly- 
Sayre materials), 

ft Mechaniral En^neering 390R.1 - Applied Statistics (reseat) - Gerald R. Warier, 
7. Radio TV'F'fm 341 - Audio Environments (rey^at) - Robert 0. Brooks. 

SUMMER 1972 

1 . Oemistry 3S4 - Physical Chemistry - Or, ^n M, White, Associate Profe^or of Chemistry, 1 19B Chemistry 
Building, 

2. Communications MO - Communication Research Design - Dr, Jat* L. Whitehead, Associate Professor of 
Speech Communication, 214AS|^ec^ Building. 

3. Electrical Engineering 360L - Digital Systems Engineering I (rep^t) - William R. Mrion (using Roth's 
materials). 

4. Engineering Mechanics 306 - Med^nics I - Dr. Wallace T, Fowler, Associate Professor of A^ospa<^ 
Engineering, 227 Taylor Hall. 

5. Library Science 340 - Basic Reference Sourres (repeat) - Linda C. Suhexnaydre (i»ing Kiii#itly-Sayre 
n^terials). 
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f ^LL 1972 

' ! ' ' CiMtiiuitation freiieat) - Paul E. Nacu/y. 

' i ^ Oi'fH»jt> VVluton E. Win iamson {using Fowter's materiats). 

3 A<tni ,oi -v L • ^ .,. r,v fwS^fi Sdi?nct» tWcijofs - Dr. WHItam H. Jeffefys, 

PwAi^^ni Asr;uru' ?n' PMyssrs Mjfh Astronomv Building. 

4. Astruno.nv 3S7\* Ljtioi»u v Motmuis «n AstronGfny - Or. Robert Robbins, Jr., /^^iate Professor 
of A^tronort^V, 17,21? Physics-Math Astronomy Building, 

5. CHemical Enymtier ing 202 Introtiuctiun to Ei^inea-Jng, Dr. Eric NutaM, Assistant Profe^r of Chemical 
l!n:iii>4?orint|, 206 E. P. Schoch Labor jtor res Building {using Na<^zv'smat6riaisK 

6. Comnn;rKC;itMm$ ,183 CMfnoiunicatfon Research Desujn {repeat) - Dr. Michael R. Chial (using Whitehead 

/. Electrical Eno,ri?«^inf| 360L - Otgital Systems Engineering I (repeat) - Charles H. Roth. Jr., A, J, Wteteh, and 
Siephen A. Ufnierwood (using Roth's matertais). 

8. Library Science 340 - Basic Reference Sources (repeat) ^ Ju!ie Bichteler and John R, Wftieai (using Kn^tly* 
Sayre materials). 

9. Mechanical Engineering 364L - Dynamic Systws Synthesis - Or. Lawrence L- Hoberedc, A^istant 
Professor of Mechanical Engineering, 41 8C Taylo< Hall, 

10, Mechcjntca! Engineering 390 R 1 - Appi;ed Statistics (repeat) - Gerald R, Wagner. 

11, Phy&^cs 416 - Elertr^city arnl Magf>etism (repeat) - Or. Austin M. Gleeson (revision of SimnK»ns' materials), 

12, Rc^dio TV Fifir 341 - Su ite Environments (repeat) - Robert 0. Brooks, 

SPRING 1973 

1, At'fospjct' En^ineermq 31 1 - At-rosp^ce Engineering Computation (repeat) - Or. Paul E. Russell (using 
Naco/y's nidterial'i). 

2. Af^rcispace Eno^nwrinq 265 - Stnjctural Dynamics - Dr. Roy R. Craig, Associate Profe^or of Aerospace 

En<j!fv^jrifM|, 5uj f Mq«nfrt'nrig Sctcfice Building. 

3. Aerospace Emji v^niMf: 367K F*i<^t Mechanics (repeat) - Walton E. Williamson (using Fowler's materials). 

4, Astronurny 308 - Descriptive Astronomy for fSlon-&:icnce Majors (rep^t) - Dr. Michael Bre^^r, 

b. Chemical Engmeering 202 fntrocluctfon to Engineering (rejwat) - Dr. Robert S. Schechter and Mr, A, T, 
Kott (ui»ing rdaco/y s materis^ls). 

6, Electrical Enginpfiring 360L Digital Systems Engineering J (repeat) - Of, Stephen A, Underwcxjd and 
Dr. 0 G, Raj-Karnr Ujsinii Roth''? materials), 

7. Engineering Mechanics 306 - En<iir>eefing Statics - Ors. Waliaa? T. Fowler and Paul E. Nacozy. 
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8. Library Sci4?nc«* 340 8j%u Infnfnrjfu* . , if i • itJ J il Bu titrU^ ami John Wheat 

Knightly S^ivre matt'iu*isi 

of Library Sen* net?, 4.224 Htimaniin:^ f^!'># aii f» (Ii iUt-f . 

10. M«::han«ca< Sngmeeriitg 364L - Dynamic Systems Synthesis (repeat) - Or. Lawrence L- Hoberock. 

11. Physics 416 - Electricity and •lagnetism {repeat) £k. Herman Wolter (Gleeson's reviw^d f^iertaisL 

12. Astronomy 383 - Fundamifntat Astronomy (G<3duai» Course) - Or, Wdliam H, Jefferys, 

SUIMMER 1973 

1. Ekjctri<:al Engineering 321 - Electrical Engineering Latn^ratory I - l>. Charles H. Roth. Jr.. ^ofessor of 
Electrical Engineering, 510 Engineering Science Buitcting. 

2, Electrical Engineering SfiOL - Dtgitat Systems Engineering I (repeat) - Or. Stephen A, Szs^nda (using 
Roth's materials). 

3* Engineering M^anics »)8 - Engineering Statics (rer^t for PSI Summer Institute for Community Orilege 
Teacht5f s) — Wallace T. Fowier and Paul E. Nacozy. 

4. tibtary Science 340 - Basic Information Sources (repeat) - Linda &:hexndydre and Charlott Roadi 
(revision of Kni|^t<y-Sayre matenats). 

5. Library Science 351 - Bas^ Cataloguing and Classification (rei^at) - David Shinder and Maurice Leather- 
berry (using revision of Herring materials). 

FALL 1973 

1 . Aerospace Engineering 31 1 - Aero^oace Engineering Computation (repeat) - Dr. Paul E. Nacozy. 

2. Aerospace Engineering 3671V! - Flight Mechanics (rei^t, being given to three students as imJef»ndent study 
course) - Dr. Wallace T. Fowler. 

3. Architectural Engineering 362L Design of Structural Systen^ tn Timber - Dr. David W. Fowler, Associate 
Professor of Architectural tn^neering, 310 Taylor Hall. 

4. Astronomy 108K - Practical Astronomy (Laboratory Course) - Mr. Tom Connell and Mr, Alan Kiplinger, 
Teaching Assistants. 

5. Astronomy 308 - Descriptive Astronomy for Non-Science Majors (repeat) - Drs. William Jefferys and 
Michael Bre^r (thre** sections). 

6. Astronomy 367M - Laboratory Methods in Astronomy (rej^at) - Dr. Paul Vanden Bout. 

7. Chemical Engineering 202 - Introduction to Engineering (repeat J Dr. Robert Mills {using Nacosy's 
materials). 

8. Chemical Engineering 376 - Process Analysis and Simulation - Dr. David M. Himmelblau, Professor of 
Chemical Enc^neenng, 21 lA E. P. Schc^h Laboratories. 

119 



9. Eiectricai Enginetsring 321 - tlectrtcai Engineering Lafaoratuty I ^repeit) Dr. Charl^ H. Roth> Jr. 

to. EtectriGdl Engineering 360L - Digital Svstents Enginrariiig I {repeat) - Ors. Stephen A, Szygencta and 
SiKptwn A. UmterwcKKl (Thret« sections) (using Roth's materiats). 

11. Engineering Mechanics 306 — Engineering Statics (repeat) - Drs. Wallace T. Fowler and Paul E. Nacozy. 

12. English 301 - Freshman English ComrK^ition - Dr. &Jsan W, Wittig, Assistant professor of English, 
4 Cathoun Hall 

13. Genera! Engineering 304 Introduction to Engsneertng Analysis (precatcuKis n^thematic^) — Dr« Gerald 
R, Wagner and Mrs, Nancy Hamilton, 112 Engir^ering Laboratories Buitdtng. 

14. Litnary Science 340 - Basic Infornuittcn Sources (repeat) - tincte Schexnaydre and Jutie Bichteler 
(revision of Knjghtiy'^yre materials). 

15. Lihrarv Science 351 - Cataloguing and Classification (repeat) — Mrs. Billie Grace Herring. 

16. Mechanical Engineering 336 - Probability and Statistics for Engineers — Or. Gerald R. Wagner, Assistant 
Professor of Mechanical Engineering, 201 B Engineering Laboratories Building. 

17. Mechanical Engineering 361 E — Nuclear Reactor Engineerirtg — Dr. Billy V. Koen. Associate Professor of 
Mechanical Engineering, 415 Taylor Hail. 

18l Mechanical Engineering 364L - Dynamic Systems Synthbsii^ (ref^) - Dr. Lawrence L- Hobcrodc. 

19. Mechanical Engineering - IntrcKluction to Operatiom Re«»arch - tk. Charles S. Bei#itler, Profwsor 
of Mechanical Engineering, 201 E Engirieering Laboratories Building. 

20. Mechanical Engineering 3% R,1 — Applied Statistics (Gradi^te Cour!^, repeat) — Dr. Bernard Most (using 
Wagner's materials). 

21. fiadio- TV-Film 321 K - Introduction to Image and Sound - Dr. Robert D. Brooks, Assodate Professor of 
RTF. 21 OB SiKidl Work Budding. 
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COMPUTER BASED HIGHER EOUCATtON: A FIVE-YEAR RESEARCH PROGRAM 



Duncan Hansen 
Memphis State Univeisity^ 



ttyiTBODUCTION 

If the C Al researchers who began the mtttal systems in the eariy ISCKI's Iwd envisioned ^e contents of a 
current tec^ntcat report, the ima^ of the table of contents wcHjfd have centered atout '"the computer as the 
teacher ." The group mind set was framed atout the vision of enrich^, ctetaiied^ student-commits interaction with 
tean f it ii| control via optimisation. Now, in the ti|^t of a ctecade time tunnel^ CAI investigators can report t*ieir 
princtpat finding: the c<»itext of cumputer-based training is broader tfian our original comreption and profits best 
from a mana^ment model for instruction. There is no singular way to indtvidualize instruction to its optimal level 
without employing some of the older techniques, surh as group discussion, or the newest ones, such as muhimedia 
split-screen presentations. This requirem^t for a broader muitif^t approi»^ to learning resulted from cnir ^ing 
researdi as it evolved through stiKly after study towards ''the ramjcaiter as the mang&f of instruction." Using a 
computer-m2"2ged instructicmal (CM!) model toencomp^ computer-listed instruction (CAIK $imulat5on 
adaptive testing, natural language dialo^es, media managemmt, scheduling, record keeping and evaluation, the 
potential of each of these components as training processes became most enhanced. This paper wtii report some of 
the research developments within this CM! framework. ■ 

for purposes of review, the studies were organized into four categories: ieamer strategies, training strategies, 
validation strategies and computer systems strategies. In learner-oriented research, investigators sou{^t to find the 
students' co^itive and personality processes reflective within computer ised training. Therefore studies included 
rule learning behaviors, behavioral do^tive learning, memory, subjective or^*U»i',un, anxiety and curiosity* 
Those involved in training strategies studied effectiveness of computer- managed instructicm, die Fusibilities of 
adaptive testing and modeling and the uses of simulation and information retrieval systems. The focus was on the 
design, implementation and evaluation of CM! components. The aria of validation studies led to the evolution of 
an interdisciplinary team, w important ideational contribution in its own ri^t, plus the role of CAi/CMi in 
schools and DOD training. Computer strategies produced, oyer the years, insis^t into inter^tive systems, instruc- 
tional ntanagement systems and data analysis systems. 

a ought toother, the strategies, the studies and the results lead us to repeat here, and throuj^out this 
report, Thost? th**»^»»< which have led to profitable outcomes. 

1. A model of computer-managed instruction yields a sufficient, integrated and cost effective approach to 
learning/teaching. 

2. The individual difference variables reflective of the behavioral processes of memory, anxiety, curiosity 
and rule application are critically important to a CM! adaptive training n>odel. The pr\rT\ary f^search need 
is to extend the set of available indict reflecting training in learning behavioral objectives, graphics, 

etc. 

^Tho foseafcfi descrth^ accompli«hod at floficla State University, cltAtlont tuch as refer to Technlcaf 

Memorandums tssued to inform the cofitr actor of t^e t^ork ttint ^es done. 
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3, V^e sekTtiiHi dful ifUeiiralion of media coinputitaits and th« reqiiireiiients for opltmal resource alloca- 
tion of iraifiirig elements (instructors. smiul*itors, peer instructors, etc.) has received limited research 
dtt4'ntH>n and rt?prt»Sf>in5 uni^ of the fi^omisimi areas of th<» futur*?, 

4, Till} rtHimremetit to build substantive liaison nuKlels between the educattwal system's leatlers and 
university researchers is essential if the tlesign chaiact^istics and requirements of training on the one 
hand ami the iheore ,ical concepts on the other are to nier^ in productive fashion^ An interdisciplinary 
research team was an exemplary mocfeJ of this liaison procncc. 

5, In refer Cfice to computer stiate«jies, the creation of managefr^nt or central prc^rams was hi^y 
profitable anil the need to create training-oriented nnwrtagement systems is of the highest importance. 

We turn now to the sf^ific elements and studies within the broad four strate^ theme approach. Each 
topic will Uu ruviewetl and summan/ed by valid CAIA3WI nmdusions. Statements as to needed future activity 
wiH fK» fTMdi? topic hy ?«mmc. 

LEAfVVtER OftfEt^TED RESEARCH 

Tht? prtrrtdry focus of CAI investigation for the tearner strategy research was devoted to studying the internal 
cognitive an<j ^wrsonality processes of the learner while in computer-based learnina tasks. Studies of rule Jeaming 
iKjhavior, anxiety and curiosity ar^d memory lead to positive indications that these searches should be continued, 
Ftesults of research into the rnle of t^ehavioraf objectives in learning and into subjecoi^ organi£ation have led 
investigators to feel that these are less rewarding areas. 

Rule Lear mng Behaviors 

Prolwbly the most comnx^i type of learning undertaken t>y students is tf>e accfuisiticKi of principles or rules. 
By learning rules in a spt?cific situation as one of a class of situatic^is, the rule t^verned t?ehavior of the {earner 
permits more effective learning strategies that «^n«rali/e to more than one situation, focause the learning and 
application of rules has tieen recogni^tfd as a very important component of education in general, a substantial 
amount of research is currently underway by several instructional psychologists. From tfits research, several 
gi?nerali/ations am t>e made as to the composition of instruction * urnpiex rule learning. 

fn this discussion of rule Itfarning behavior, it will be Mpfu\ to consider three interrelated concepts. These 
are the rule statement, the rule and rule governed behavior, A rule statement d^ribes the prcKredure to be followed 
in iwi f Of mif^ti a spi'Cif ic ojieration on inputs from a sj:^cified class of inputs to produce a specific output from a 
class of outputs. The rule is considered to be the procedure or otwation describes! hy ^^e rule statement. Rule- 
governed behavior is that Iwhavior that would result from a student correctly applying the rule. Tfieambi^ous 
term ''rule learning" may now be divided into two appropriately descriptive terms: (a) learning the rule statement 
which refers to It^arning to vurbalue the rule statement, and (b) acquisition of rule-governed behavior which refers 
to fhf' correct application uf tht* rule, 

A fcpiicdliun stucJy was tmplemented to investigate prior findings ttiat rule statements pre^nted in the 
instruction reducefi fhe numlwr of examples neetled and total time required to meet a prespecified criterion. 
Further, presentation of rule statf?ments increased performance on a transfer task and reducetf the requirement of 
reasonsing ai)il;tv in l^'^rnmg the task and presentation of objectives also reduced the number of examples and 
reduced the ri?qufrement for reasoning ability. The replication showed that presentation of rule statements 
riMiticed the ntimliHr of example refftnred to meet criterion performance, increased posttest performance, 
reduced tottil time rf»f|iiirefl to meet criterion performance, increased refontion fx^rformancu. and reduce<l the level 
<*f statf* doxiety wifhin th<» iHarninq task {T1VI59). 
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A third study in this area found that the iKesentation of obfective^ increased the total amount of time sp^nt 
studying a rule learning task {TM47>. The iKes^tatton of sc^pie test items within the learning task redum) the 
rcjiiuireinent for memori, but neither obiectives nor sampfe test items had an effect on post or retention performance. 

impiiuations from these studies for instructional method in acquisition of rule appiicati<m skitfs appear to be 
thre<» fold: 

1 . Instruction should present general instructional objectives to inform the student as to what is ex^^Cled 
of him. 

2« Presentation of rule statement can prevent the ''discovery'* of an incorrect rule. 

3. Presentation of sample test items gives the stuttent a charts for pr£K:tice and immediate fe^t^ck. 

f^urther studies revealed that availabsHty of prior examples reduced problem respond latency and reduced die 
tequircmeni for reasoning alMhty {TM in press). A present investigation, current ty being analy^, apiK^rs to show 
that memorization of work statements redut^s both the numb^ of examples and total time to reach criterion 
performance on complex rule application tasks. However, no memorisation effect has been found on the pcadttest or 
retention test of rule application skills, nripiications for instructional development are (a) the requirement of 
memorization of rule statements to facilitate the correct choice of a rule when more than one is being teamed, 
and to aid in recognition of operational symbols if they are used: ^b) the use of several e)»mple$ to ^ow the state- 
ment instances of th*? correct application of the rule; and (c) the requirement of reviews if more than three rules 
are being learncNi in one sesiion to provide simulated testing and apfs^opriate followup review instruction. 

Future 

}n addition to apparently useful generalizations given for tiie dewlc^3ment of instruction for ^quisition of 
complex rule application skills, impiicati(»is for fruitful future research in this area have grown out of the FSU 
investigations, it shouki be ascertained wfiat type of examples are most profitable presented to the student: norn 
examples, examples, or both. A second line of research recomnwnded is tfie investigation of desi^ of rule learn- 
ing materials impendent upon the abilities of the learner; this vw«»td mntribute to the adaptive model r^arch. 
A final area which appear to offer promise is the area of optimal number of examples for presentation before 
sample test items. AH of tnc»c studies relating to number of examples would contribute to a drill and practice 
adaptive model. 

Behavioral Obfective Learning 

Only in the latter part of the decade was research undertaken to examine expiricaliy tl^ dalms made for 
obfectives in instruction. Of the approximately 35 studies conducted, about half failed to confirm tiie hypothesis 
that providing students with objectives leads to increased learning. Remaining studies showed facilitative effects 
(TM45). The research on interactions tetween objectives and type of learning provided few ^itive findings. No 
general conclusions can be found in the research whidi f cKJnd a few significant interactions tetween ob^ctives 
afid learner characteristics. 

Investigations at FSU found that c^jectives si^ificantly ina^eased study time with no posttest difference 
between groups (TM61 ), and that cAjjectives partially reduced anxiety without affecting performance <TM47), 
The hypothesis that objectives ackl structune to learning as is done by advance organizaers was not confirmed, 
although a n>ain sequence effect was ^t^nd {TM57). A further FSU study found that objectives focused learning 
on relevant materials and depress^ incidental turning (TM66). 

To generalize from studies overall, it seems that objectix^s can have an effect on learning, although this 
effect does not appear to be as strong or as pervasive as originally assumed. 
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future 



It IS anl4cif>aie<f that research on t^haviorat oh^t!ves will continue the trend away from eKploratton of 
tficsi ef on If urninq. Tht> it\u%\ wiH t>e toward exploration of interactions with task and learner characteristics. 
The explication of hypothesized cognitive functions which may be fulfiHed by <^|ective$ m the le^ntr^ situation 
is anttCMiatetl to qrow, ^so. An examf>le of this tyue of research is the tn^trgatlon of effectives as an orienting 
function (TM57K Furtht^r research in these areas may prove more productive than that amducted to date. 

Ritemory Research 

An im;»rtant capacity for memory is the ability todtscriminafe. This is an idea v^ich has been reinforced by 
the research which we have comlucted on directed for^tting. Important to in;: retrievat of some previously stored 
event is that we be able to discriminate it from other stored information. We think voluntary foi^tting is largely 
a mattuf of sharpening one's capacity to discriminate between material whicb is to be remembered and that v^ich 
is not (TM19}. 

This way of kxjking at thinfs raised the c^estion of the future of memory stora^. If one's ability to dis- 
cnmuiate amonq memories is important, then perhaps encoding is in terms of dimensions which offer a basis for 
discriminating amoty^ memories, and more specificaity, in terms of the valims of such dimensions. Just tt*iat these 
dimensions are is unknown. The point is simply the prooosition that events are encoded according to abstracted 
features. 

Another study (TM in press) investigated short term memory (STM) prerequisites to development of more 
comples menta! processes. Of Sfiecial interest was the possible dif fer^ces tetween white, advantage children 
IB-? years old) and Wack, disadvantaged children. Differences would seem to indicate that memory characteristics 
are not thi* $dme for these two groups. 

The f indmgs were also compared with prediction of three of tte more characteristic infonratron r^-ocessing 
models for STM; ^^rling's Visual STM M<Kiel, the Feigei^jaum-Simon Computer Simulation Model, and the 
Atkinson Sdiiffrm Model It was found that some of the results could be inasrporated into the structure of a(l 
the models, but that no one model could account for all the findings. An alternative model was developed which 
incltklec! dn IQ parameter, a more complex rehearsai process, and an oji^rational representation of organizing 
strateqies, ^ 

Our conclusions for tioth the directed forgetting and STM childhood research are that they both indicate 
fjossihio ind'vtdual {lifferences which could be useful in adaptive instruction. It is necessary to continue research 
to fully operationahze appropriate memory indices. 

Future 

In the near future of memory research, the computer will be used to construct theories and to simulate 
memory pL^rformance rather than to assist in the conduct cf experiments. This will ^particularly true in the 
areas of semantic and long-term memory. These theories are particularly complex and it will be especially useful 
to simulate performance. By contrast, expenments to test such theories may simple and fairly easy to con- 
duct with relatively unsophisticated equipment. 

Subjective Ogant/ation 

The purpose of the research on subjective organisation (S0> was to determine if SO is a meaningful individual 
difference variaWe in the learning of veihal discourse. Furthermore, we thout^t to examine the relationship 
brtwi^en SO and other aptitude variahlos thouqht ♦o be important determiners of l*?arniruj from prose matf^rials. 
In order to accomplish this, a new n^asure of subjective organization had to be developed because previous 
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meiisure haiJ c&rUm limiutions vwhich were thou^it to mitigattj aqarrist finding a meaningful relalionship between 
rmmiorv organi/aiion <ind ft*calK Thus, ats an outcome of this research, a new measure has been examined and 
Piirtialiv valtdateil in relation to other SO measures and other individtial difference variables. 

The t>asic research paradi^ was one in which SO and other measure were administered to subj^ts and 
tht!n those subiects were asked to learn paragraphs which \fme organized in various ways. The results indtaated 
that hii^i organisers are highiy influenced by the external structure of the learning materials and they tended to 
mirror that or^nization in their recall, Mit^ organizers, however, wpr#* able to reon^nize the materials, but that 
was not to their advantage because of the time constraints involved. That is, hi^ organizers actively sought 
atternattve teiationships among the sentences, thereby reducing the amount of time spent memorizing them. 

Three generalizations can tie made on the basis of the studies which were conducted at PSU on sub^ive 
organization: 

1 . ft seems that the ability to reorgani^ material can be a limiting factor to a learner if sufficient learning 
time IS not provided 

2. Low organizers need to have learning materials structured to a higher degree in order for them to 
learn effectively. 

3. Ht^ organizers perform equally on materiab of hi^ and low stn^cture. 
Future 

The conclustoris of th*- investigations in this area indicate that future resea»th should consider the following: 

1 . A fM^actical, reliaWe, and valid instrument for measuring SO must be <tevel<^}ed. In this regard, the 
potential use of a computer for realtime pre^ntatton and scoring is promisirKj, 

2. Explorations on more complex instructional t«^ks must be undertaken on the fimling that low 
organizers need hi^ly structured materials v^i^as hi^ CKganizers perfonn the same on both high 
and low structured materti»ls. 

Contmuatlon of research m those areas should provide instri^tional desiyutus with a set of findings which 
would quide the structuring of textual materials such that the learning of all stud^ts will be maximized. 

Anxiety and Curic^ity Research 

Our mvesti^tion of anxiety and curiosity behaviors of a student while being trained was primarily focused 
on the use of these personality ifxiices as predictors of optimal instructional treatments. While State/Trait Theory 
initially guuled our efforts, the subsequent inconsisterK:ies tetween replirataWe finding and the theory irxlicated 
Its limited role in a training context. The theoretical study of curiosity did provide us with a new framework which 
appears to resolve many of these prior unexplicable a>mpiexities. 

The developmem of State/Trait Anxiety Thfeory, based on Drive Theory, allowed a number of derivations 
for prediction of rjerformance relevant to computer-assisted instruction {TR6, TR7), At the san>e time CAI 
seemed an excellent vehicle for testing the theory {TR14), The results were not encoura^ng for confirming a 
Drive Theory interpretation of State/Trait Anxiety. However, anxiety was shown to be an indicator of CAI per- 
formance and the thrust of research continued in this area (TM20, TM41, TM42, TM49. TM52). Studies irwtuded 
annieiy effects on memory support, test taking, and mter^ttons with response motfe, subject matter familiarity, 
and learning time. 
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The impottam faults indicated a consistent inverse relationship between state anxiety and learning diHer- 
enital effects of traif anxiety especiaJly as it revealed learner latencies, the atmiHex role of memofy aids and 
response motfes as facilitators under some conditions, and the contpleic functional relationship of anxiety 
ctniositv atui fatenctes. Th« latter f imling could be a promising future framework for studying latencies within 
training. 

In adtlition. the anxiety theory and research directly initiated and influenced work on stateAf -it 
eptttemic cunostty <TR23, Curiosity provides an additional indicator of perfonmnce within instruction 

arid speafKraUv C«l. Ihe concurrent inie.ests in adaptive instruction models resulted in postulating the value ot 
these perso,«htv variables as useful predictors in decision models for selecting alternative individualized treatments 
One stwfy, unlwmg regression techniques, espeaally found state anxiety to be a useful predictor for sefectina 
remedial instruction (TR27}. r 

' \ 

FiiUne I 

For the future, we anticipate that the effort within adaptive models and the theory of anxiety curiositv 
wahin traminy will be highly fruitful. The use of anxiety-curiosity concepts to study response latencies miy hHo 
explain tht- highly variable results one tenrfs to find. 

Gr^htcs Research 

in nrrtcr to employ the r«i-cjich capabilities of computer presentation of qraphira m instruction, an initial 
study at FSU provided the opportunity for devising teeh^nues to ccMfe graphics onto the cathode ray tube screen. 
This general model of graphic encoding was then used to develop studies of selection and sequencing of graphics 
for realistic training materials(TM14). Further attempts to iearji the rules of graphic learning <TM24) demon- 
strated that the search for such rules is difficult, and can even 6e consittered unprofitable. 

Furthering the graphics investigations, FSU pursued research in mapping sentences on to graphics (TW60). 
Studying this insUuctional technique provided data which was complex. More aj^ared to be going on in this 
instructional situation than could be easily int^pret^ by learning theory. 

Us«rt as a teachinq aid mmputer graohir-s nro«ed helpful in instructing engineering dynamics students in the 
traditiorwl «um«iing hlock -^f Pu!er angles (TM18). Graphics proved helpful in assisting in wisuali/ation „f ihe 
angles. Suliject;, also reported {jositive attitudes toward the instruction. 

future 

Generaiiy, given current constraints, the search for rules in learning from graphics ainwars to be unsuitable 
for stutly. As an aid in problem solving, graphics apparently lend assistance to learning; the experience can be 
positively vieweil by learners. 

TRAINING STRATEGthS 

A majof investigation of computer managed instruction was conducted as a priority in training strategies, to 
study training effectiveness, and cost Ijenefit outcomes. Further training strategies in the areas of adaptive testing, 
adaptivw systems, simulations, and information retrieval systems were also researched to various degrees. The 
research, qe^erall^atlons, and future trends ar<?^descritwd in the following sections. 

Computer-Managed fnstrvction 

Thtcniqhoiit the i((fr.ition of thf contact lh»;f« was a concurr»,'nt set of investigations m computi?! -assisted 
instruction (CAI) ami com^wter-managed instruction (CMI). The primary purpose of llw investigations of CMI 
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to detetmiim iis Iratmng effectiveness am! assc^iaied cost bemhi outcome in comparisoti yvah CA f afHi 
other niore convent lOHdl meamof instructkm. As pursued at FSU, computer-manacietl imtf*;ct«on loviilwi itn* 
fuilowiny: 



1 . Uiagnost^c assessment ami the assignment of individualizetl tear nmg |»rtJ5Cf ipt wm, 

2. the use of CAf for iH^acttce and remedfattonat fiur^Kises, 

3. the use of simuialion for rote ami dects^on-makingtf;^*'^^**^ purposes, 

4. thi? use of thfi cnmpufi»f for easL» and objectivity of ctiitfculum devefopment. and 

b. the development of a record system so that the imtivtduafixed training process could 1^ ef feaivtlv 
monitored and maiKiged. 

With this CHvf I conceptual context, a number of studies ^re porsueti (TR8, Tfll 1^ TR12, TR22, 
TM23, JM32). All of these studies indicate that CMI at the colle^ate level is highly feasible, cost effective and 
^ifnvide?i for learntntj results similar to CAI. However, liue to the mastery level learning apf^oach utilued m the 
ifistfuctional materials, the relatton^ip of individual difference variables to learning rate or performance 
fnure limited. Where extensive media and recitation sections were used, the effects of mdividt^l difference var la 
Nt?s seemiKl to be more pf onounced- Finally, learning attitude toward the instructional materials was quite 
positive* aiul could be manipulated by the form of training* 

In twn, investigations of CAf indicated that it is useful in a number of technical training areas. It proved 
esptu:mHv useful tor dynamic graphics such as found in engineering dynamics, {however, while CAf was shown to 
be viable in areas like chemistry, the results did not tentf to exceed those found in CWI. 

It is important to note the fact that the development process for C3Wf , while not quite ^ demanding a^i that 
of CAI, still wufi considerabli?. TfH? tleperKfency on a sound training modef, formative evaluation, and effective 
innnitot mi| of students in an ^n^^ividuah^ed mode seem to lie the critical factors in the design and implcmt^ntatinn 
uf CMI, 

f utm this, conse'^tiently, the foHowinq f^search tjener -^^^^^riuns ran be derived; 

1, Tormifial oriented comfniter-nuHui^ instruction has shown to be more effective than conventional 
instruction and less c»stly than computer -assisted instruction. 

2, The rmist significant gams in the ffuahty of instruction have not nece^arily t>een due to the use of 
computers, but have tieen through the iin|>}ementation of systematic approaches to the trairtmg 
proci-ss required for application of the computer. 

3, Although the computer provides th»f instructional de\^loper with more information about the 
instruct lonai process than has been available, the revision process remains the least welt understood 
and utili/ed component of the systems approach, however, fhtf provision for systematic, reliable 
data now allows us to turn our atti^ntion to this problem, 

f ■ 

4 InterdKCiplinary collegiate developnrent teams wilt not necessarily produce bener computer i/ed 
instructional materials than thosr produced by conceptually integrated teams. 
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future 



During the time pertocf since the mttfation of tt^ Tt^esnis Project, educatcffs haite come to understand that 
in ortier to lmp!ement imtivkfualiJted int&truction fw the learner^ there is a need to i^ovide stmi>ar tratning for the 
ttmncf. This new form of teacher training has been labelled *'ccHiif»teiKry4iased" or ''f^formancs^based'' train- 
ing. The itniriementatkin of such trainiiwi prc^^s has more clearly icfentifi^ the mmh which sudi programs 
have for a computer* These con^ter needs, in general order of i^icn'ity, are Ifst^ below: 

1 . Record Hewing of oomputeriared and roncomputeriaied ds«tssn^nts of student performance for 
monitoring purpom 

2. Scheduling 

a Assessment of "computer-t^^e" skills 

4, Adaptive remediatic^ Jncluditty CAI ami drill amf pri^i 

If this list is an awurate reflection of the nrcds of sudi f^ogprams^ then it would appear that the role of the 
computer must be first as a ^neral mana^n^t system and only secomlarily as an a^essment and instructiorial 
tool. 

Adapth^ Teslirg 

The initial uraier taking with adaptive testing concerned itself wi^ the autOTrtation of inteBectuai and 
P rsonality measures. This effort suoc^^Hy ratted in the automation of an imKvkftAlized imelligence t^t 
(Slosson tntetligepce Test) ITR21 1 with sathfactory relfability and validity coeffkiems. Moreover, a complex 
pefsonaHty test, namely, the MMPI, was also satisfactorily implemented <TM28« TM48K These studies imf N»ted 
,that respoi^ latency amtributed to tte score informatkm ]^ conoirrent validity when n^smed ^imt a 
tomentionai administration. Th^ ifne of research et^tuated in a systerratlc review (TM3« TM30) amvering the 
v&nous uses of automated tiding. 

In turn, the dcveiopn^nt of sequential testirq and tailored testing continue to be investigated. Utilizing 
simulation t^niques, sequential testing can be shown to be a fi^ofitaMe a|^>ro^* The ig^r^ goal of sequential 
testing k to present the sn^U^t nuniier of test items necessary to accurately classify an examinee into two or 
more exchi^f e groups, or In com^^ptu^ tem^, to inwove reli^ility and validity of or the d^ir^in psfot^ within 
C Ai testing. Previous comi^iter-lHised s^^uemial testing huKlels have b^ k^ed upon r^it^wrong mea^ir^ of 
performance. Die FSU sequential testing effort differs from previous models in that both invartate and multi- 
variate {^f<»tT»nce m^»ires ara utilized. The nKittivariale fnedictitm model is expo^ to inaea^ c^^^fication 
accuracy beyond that possiirte using binary-coded item r^pon^ data alone* More intportantly, the u^ of t2»ior«t 
testing is currently t^ing investi^ted ^ an appro^ h:>r the administration of attitude iten^ in a training situation 
|TM in press). It is conjectured that the nun^ier pf test items can be minimized and the assessm^ of learning 
satisfaction can be improved in terms of its accuracy* 

These studies of sec^ential tt?stinq led to the demlopment of adaptive reading o^mjM'ehension tests. The 
adaptive reading tests provided for a mere dyn^id imJividuaJized ^rch of the student's oirrent oomprrfiension 
level especially under paced conditions. The rationale for using p^ed conditions is t>^ed on learning-time saving 
being pactf continent. 

Future 

In the future atfaptive testing v^ai most likely focxis on two a^iects. First, an investigation of incorrect as vrell 
as correct alternatives via Bock's multiple cate^ry mcKtel shows great promise. This approa:h should allow for 
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adiH^iive setecting of items by items alternatt^ characteristics and the assicpmiefit of sequentially cterived c^n- 
ffdence fodfTds about the student's ptH-fc^rmnce ievet* Given the improved information from this atH^oath, it is 
then conjectured that a sampiiv^gi al^proadr to ^uentiat testing van more effectively provide for the early ctassi* 
ficdtiofi of a sirt)ject espectalty if complex confidence band techniques are utiltaced. Thus, the future tends to hoW 
a continued peaking for more efficient testing procedutes as weif as ones which wiit have psychometric properties 
j^f^opriate for the individual and not |ust the crcaip. 

/klaptive Atodding 

A mapr theme throuiyhout the ONf^-sponsored research has been that instruction should not be fixed for all 
students either in treatments or tinw. The alternative to fixed Insta' Jtion is ad^^tlve instructlw which attempts 
to match an individuarsuniqi^ charmrteristf^ with c^timum treatment dimensions. Modeis for sheeting aiH^opriate 
alternative treatments consist of contintjait, mediated, and optimtzii^ algorithms designed to achieve tfw 
objecti«/es. 

The inttiat studies of adaptive miKtels at FSU investigated teamir^ and pa^onaUty variables which might prove 
^ useful in predicting and assigning remedial instruction, iesscm length, and otha" instructional ^^les. Muitipie 
linear recession methods were developed <TR9| ar^i used in an early nKKfelir^ attempt with predictor var taU^ con- 
sisting primarily of (^formance variables iTRIOK f^rsonatity variatiles were added, in a pha^ aiH^N^oach to deter* 
mine useful predictors, which result^ in caitstanding performance levels for the adaptive group. A mafor study whk^ 
manipulated the aisigpiment of remedial instruction was undertaken ITR27K The order of outcomes in terms of 
Superiority were the adaptive model strategy, remediation for aH strategy, a learner chc^e strategy, and a no> 
remediation strategy. 

These results and models were then consictefed for employment in reading cswnprehension (TR20K More 
recently, models t^sed on the experience gained in the ONR-spomored re^rch, were developed for tl» Air Force 
Advanced Instructional System. In ^dition, ONR adaptive modeling has continued with an investi^tton of drill- 
and*practice in terms of lesson structure composition. Further, the study of the utility of measures which relate 
information processing to reinforcanent OMitingencies for adaptive instruction has resulted in a model which takes 
into account tioth memory ami motivational prtwresses and proredures* 

Future 

It is anticipated that investigation of the monitoring and mana^ng aspects of both training re»>urces ami of 
the individual will provide payoff in the future. This will invovie rr^naging such things as the f^^cdtilem set sire, tlw 
sequences among sets, the assignment of media, the assignment of instructor and peer time, and the use of inmitives, 

Simulattors 

System Simulation 

One of the maior methods of using simulation in education, as well as other spheres, is the jKojection and 
analysis of systems. By simulating a prototype computerized n^nagement system intended as a suba>nTponent of 
the FSU elementary teadier training model, the CAI Center examined a number of problems in a data acquisition 
and instructional system ITR8K The following ^neralizations resulted from the system analysis: la) a computer- 
managed system is technically feasible for use with an individualized teacher training pro-am, and (b) computer 
r»^cx:essing \cpt trainee scheduling and testing data, as well as program mara^n>ent, is n«»»ary. 

SimulatirKj a teai^ier training system using inforn^tion from a t^havioral simulation (TR16), FSU found 
advantai^ in experimentation capabilities resulting predictions about the teacher training program. Further, 
advante^ were found in the utilization of recession twhniques in combiration with A Pro^amming Language 
lAPL) for designmg, constructing, and utilizing a system simulation. 
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A vehavioral stmufcUion was used at FSU to train pro^>ecttve tei^hers by providing an environm^t faciMlat- 
ing transfer to the ctassrcx^m {TRI6). Analysis for the behavioral simiriation proceecfed through assessment of 
entry behaviws, coltection of data on trainee p^formance during toistruction^ €»Hection of data on criterion 
variabtes and analysis and evaluation of the recpiired training process in terms of the dimensions of simulation 
theory to estimate the nature and vatidity of the tehavic^at simulation. Behavioral simulation speared to permit 
change in participant's behavior pattens. 

instrvctionaf Simuiatkm 

A further major function of stmulaticHi is instructionat. Wt«h of the debate in this ar^ continues to be over 
the potential for simulation to affect attitudina! and/c^ ccqnitive growth. One ^me, or simyi^ion, devek>ped at 
FSU (TBI 71 Mras designed to allow the player to relate ht^ knowled^^ of science to situations in elementary class- 
room instruction. Use of the game by ^aduate stuctents indicated that a tearniv^ environment of this nature can 
loiter growth in both cugnitive^and affective areas* 

One of the primary advantages seen in instructional simulditiun is the ability to reduce cost {e«g.« by reduc- 
tion of e^per^ive laboratory equffiment) and a second is experimentation without danger (e.g., in^stigating danger* 
ous chemical combinattom without use of the checmicalK tnve^igation into simulating labm^atory expa-tences at 
FSU showed that simulation, as opposed to tradstionat laboratory methods, was equally effective as evaluated bv 
posttest performance and total instructional time. 

Statisticaf Simutation 

As is sug^ed in tbs Compute Syj^ems section of this import, A Procp^amming Ldf^uage (APL) proves be 
an outstarKling computer language for the simulating of statistical ar»lyses, due to in mathematical capabilities 
(SM3K tt is possible for a student to expliKe the meaning of various Mistical concepts throu^ sample prc^^tems 
and exercises, whtdi he works as often as he v^^es. He can be gukJed through the hypothesis testing procedure 
by inspecting data, successively makir^ appropriate infarential decisions as further hypothesis testing infonmtion 
is given him (SAd12)* This type of APt simulation a>mtnu^ to expand as its utility imd appeal are exp^>sed to 
wider audiences at FSU, 

Investigators of simulation ^neraliy predict a mc^e widespread use of the t^nique, both in education and 
in other spares, for research and analysis as well m in$tructi<Ki aiKi training. Advances in the development of sudi 
simulations, however, are nreager (TM53). The major prc^iems affecting progress are seen in e^luation: 

1. Many authors feel that their simulations produce aff^ive rather than cognitive chan^ and are 
unsympathetic to stating behavicKal objectives. Sir^ an equal number of authois and users request 
behavioral objective fcK simulation^ this prd^lem may not be resolved until statements of affective 
objectives are satisfactorily developed, and can be ^vatuat^. 

2. Evaluation of instruction by simulaliun correspondiniy difficult, A ^^rxA^ for satisfactory, new 
methods of evaluation may be a future activity, or a satisfactory relating of simulation evaluation to 
existing techniques. 

3. The validation of the simulation mcxlel itself remains the nx>st difficult part of simulation development. 
While mathematical simulations appear to have potential to solve this problem through output com- 
parisons, simulations of human behavior, for example, provide a requirement for estal>lishment of 
other model vdlitiation procedures. 
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Information Retrieve 

tfi for inquffy 

Infomation retrtevat HH) systems. Uke Simulations, can be devetoi^ and used for r^arch and/or ii^Ituc- 
lion. FSU research on a CAi based information retrieval system containii^ 53t2swidf ^tence is^rirfkations was 
directed toward enamhtation of human tmiuirv bdiavior and appraisat of affective factors within the task (TW116K 
Both the feasibiitty of the approach to improifing tr^uiry tehavior and the positive nature of ieamer reiK:tton were 
established in tfie tR application. 

fR tirstruciioni J Tasks 

In the library science area, rerognition of the future ^rcmth of aulonuited IR svstctits ted to provision of 
computer-based iR tasks for advanced students {7M40K Both imtru^onal and research sp>a{s wme t»t for tii« tse 
of the on-fine searched coordinate irKfex^ v^ich reference 18^ documents prior to its trans^ to the IBM 64(M>. 
As a teaching ^id In s^adii^te courses, the f R system «frved several imtrucliomit pmposes: (a) a ctemonstraticm of 
an ^>line search was available to stuttents, ib) the system t^ovid^ lessens in f^eparing and searc^ir^ crordinate 
indexes, (c> the system permitted students to develop skilH in the use of computerized IR. arrf (d) students were 
able to perform test searches of the index with reference questions* 

For research purposes, alt student uses provided data which were collected by the corriputer system and 
2»laied for use in IR evaluation and revision. Data coiteded on ^udent attitude were positive^ as were professional 
observations. This system was one of the first implemented in library schools. 

Future 

I 

Since good inciuiry skills are unlversalty useful, the potential of IR systems in enhancing the^ skitlis is an 
area of interest in which future research is recommended ITfWIB), It is sus^^ed that the new technkiues of 
cd}servation and ei^luation gainai in p^ticipatk>n in these developmental effects berome part of the individual's 
professional research, development, and teadiing appro£K:h, thereby enrk:hir^ members of such ^oups. 

The development and use of information retrieval systems in library sciems is an example of u^ that can 
be made of specialized IR systems in a nun^er of fields. Sim» tjte number of fields which will computers and 
IR systems to their requirwnents continiw to ^ow, the devel<HWT>*^ of this type of irtstriKrtion stUHikl be present 
in cofKTomitant mstructional areas. 

VALIDATION STRATEGIES 

During U h; FSU research, every effut t was macte to interrelate with ongoing ii^titutic^l activities, its 
associate j^oblcms amJ its need for research. For example, a team of FSU professors who were also active in the 
Navy Reserve was formed into a functionai CAI research unit, This grcHip represent^ a bridging (M'ocess between 
the various r^arch studies formed at FSU and their application within the Navy. 

Navy Research interdisctpiin^ Team 

The team of FSU professorial investigators who also had active relation^ips in the local hbvai Reserve 
pursued a nuirriber of objectives. First and foremost, the ^oup applied tfie <x>ncepts arnl fwocedures of <XHnputer- 
mana^d instruction (CMI) to local training of the Seaman 1 classification. This CMI implementation study was 
quite successful in that tN?re was both a si^ificant increase in performance level as w^l as a significant savings 
in training time. The results were tempered by the small number of participants, and I am ^re, are in the process 
of being replicated at such bases as the Air-Navai Station at Memphis, Tennessee. 
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Secondly, the group purged bfoad systems aont^pts for the ptannii^ and mana^ent of NavaJ training. 
This led them to attend many conferemres fnxnride consultation help to the Ua\m\ Rc^^Nrve system itself. In 
many cas<s the tean> members p^formed active duty, which led to si^iif icant plans whereby more effective 
reserve driMs could take place. 

Lastly, the team lirovtded leadership for the Navy conca^nlng university and comrminity relationshif^ 
As university models, they effectively contribute and are utilized Within Naval Reserve operations* Setximity, In 
re^rd to community involvement, they advanosd the Navy concernii^ stH:h ^i^iat training problems ^ ra« 
relations, drugs and other associated b^avioral nurners. Perhaps moredtrectly to computer activity, they 
assisted in the design of information mana^ent systems areJ ss«K:iated aiteptive training and testing systems. 

In regard to some constraints, the usual problems of cxHum^fcation sfftd tack of goat understanding handi- 
capped the group at many twm. Given a fefr d^ree of illation in Tail^d!^, Florida, the ^oup maintained a 
vigorous information exdiange whidi umtotH^tedty i&t to ttwi si}a::e$s«>s that it «^hiewd. 

DCH) Cdmptiter-Rdated Activities 

As an indirect consequence of our FSU resea^rch effort, the project perswnel was ^ abde to serve the 
Ctepartment of Defense on other trainii^ projects. The most significant one of the^ was the AF /Mv^umf 
Instructional System (AIS). We wuld not have de^gr^ and specified the AIS without the prior vmtk of the 
Lomy Human Resource laboratory group or our criticai CAI/CMI existence and data. We turn now to a 
description of the AfS project. 

The Advanced Instruct ionat System (AfS) is a cievetopment within die Air fnme Human Resoitrces Labora- 
tory {AFHRO to implement ttw latest demonstrated s^atenafthe^t In training UM^hniques, media usage, mana^ 
ment proc^ures, and computer technology to Air Force Technical Training. The CAf Ca>ler was awarded a 
contract by AFHRf to devefop functior^l dcsi;^ specifications for the indivMuat multimedia <ximfHfter-twed 
training system which would provicte si^if icant cost-effectiw improi^ments in the operation of technicat training 
courses at Lowry Air FcB'ce Bawe, Colorado* 

In addition to the goal of providii^ individualized training, AfS will focus on the mana^al process which 
can be enhanced by the computer, cost^ffective multimedia approaches which may provide tiTT^-saving^, mcKiutar 
implementation which will provide both flexibility chiring develi^nront and revision of learning materials and 
£MMitiofml cost savings during expansion thrc^^Hnif the Air Force. AIS will be imf^eir^nted within three techni- 
cal training courses with a total enrc^lnwit of over 2000 students, Tt^ courses of Inventory Mana^ment (inventory 
and supply), Precision Measuring Equipn^nt (precision electronics measurement and calibration^ and Wea^KM^ 
Mechanic (tactical weapons loading) r^esent a broad range of technical trainif^ requirements. 

The AIS insists of seven sut^ystems which refl^ the scof» and comf^exity of the effort: (a) imtructional 
materials, (b) instructioraf strategies, (c) media hardvwre and software, (d) mana^ment components, (e) compu- 
ter hardware, (f ) personrwl and training requiremems, ami (g) related r^uirenwnts. These suteystems are designed 
to provide for all aspects of the instructional process frcwn materials development and evaluation throu^ ^udent 
use and mana^ment to review and revision. Operational consideration.., including omiputer systems d^t^ and 
mamtensnce, CAI languages, and the selection and training of AIS per^Minei are also romidered in the design. 

Wakulla School System 

The Akulia County Title 111 project focuses cm two prim^^ goals: (^) imptomematkm of computer- 
^sisted instruction in mathematics and reading for &HJthern rural stuctents, and (2) development and imple- 
mentation of CM! the mentally retarded. Se^^n in July, 1968, the project is a joint undertaking of the 
Wakull;^ County, Florida, Board of Instruction and the FSU-CAI Center. By the Spring of 1970, SS is 9iade- 
viiie Elementary School and selected seventh and eighth grade SS at M«lart Hi?^ School were receiving two to 
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three computcf -assisted instruction lessons per week in reading and/or mathesnaties. Math fnaterlals had under- 
gone revision, and reading lesson assignm«its had been varied to best accommodate needs uf the Ss. 

As in the Spring of 1969. testing results in the Sfwing of 1970 indicate the 8s who participated in the 
CAl instruction showed positive gams in both reeling and mathematics. In addition, attitutde measures com- 
pleted on teachers and Ss involved with the comfsitw instruction demonstrated positive res^ions to comj»rter- 
assisted instruction. In Ihe Fall, assessment instruments previously us«i vme eliminate becauw it was felt 
that they did not measure the effects of the instruction; and individualimf reading tests and a aiterion- 
referenced test, constructed from sample CAI reading items, were administered by project personnel. Spring 
postte"* results on these instruments ami the regularly administered siareJardiied adii«jement test indicated 
positiv <^u(ts. 

The CAI program was the first of a three-phase effort designed to serve between S) and 70 EMR stu- 
dents in three etementary schools and one hi^ schtwi. The second phase involved the development aral evalua- 
tion of a computer-mana^ instruction pro-am in which the computer was used «ity for evaluations a«J 
prescriptions. In Phase 111, a curriculum will be develc^ied which ikm not inv^we computers. 

The Phase I materatals were prepared oo four readti^g levels. Each student's progress was monitored 
daily and each was assigned to the level suggested by that evaluation. The Irwesti^tTOS found thai the 
difference faetvaeen the third ami high«t levels was too broad, so a nsw program ««s developed and 
evaluated to bridge that qap. 

In addition, an experiment vws designed to test the efforts of picture-siimuli on vocabulary acquisition. 
The study demonstrated that picture-vwird matching significantly improved the student's ability to learn new 
wwds. 

In September. 1972, Phase II began with the writing of programmed booklets to be used in the CMI 
program. Many materials were written to eliminate *ound during the Phase revaluation. A nsw branch- 
ing technique was developed which was based oo a hierarchkal development of reading skiUs. Aslo included 
in the materials were pictur«-word matdtes suited by the study in Wase f, but the cont^t has t^n ex- 
panded to indude picture stimuli to be matched wi* }:rtionetics, i^rases, are! sentences. 

C<»<N>UTER SYSTEft^ STRATEGIES 

• There arc three main areas of design and implememation of c«nj»iter systems for instruction, namely, 
<a) interactive programming languages. |b| managemrait schemes, and (c) data analysis sytems. The software or 
i^i^age developments employed a common strategy, to maximize the ease of use white devoting a minimum 
of develt^mental effort. The topk» within this tiieme will be organized around the above three topira. 



inter get ive Systems 



A Programming Language (APL) 

The development of the programming languacp APL (A Programming Language) (W the IBM IWO CAI 
system was a project oriented towani Uecermining the capabilities of this type of lan^age for m«»ting the 
diverse requirements of computing in education. The elegant and powerful data structures and commands of 
fiSn^ appear suitable to the areas of instruction, data processing, and computer science (Sy«ems Memo 4). 

APL Gener^fty \^ 

A measure of any programming language is" its need for extensions. In the instance of APL. extensions « 
the form of functions were developed in the areas of programming aids, statistical analyses, character handlim 

r 
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numerital martipulatron. and graphic control {SM3}. The fact that such APL functions can be created and so 
extend the language is to its credit; however, the fact that extensions were needed is an indicator of educational 
requirements for more than one computer language. 

APL tnstnfctionaf Prosrmiming 

APL*5 usefulness for statistical analyses and simutalion activities {SM12^ has been documented by PSU. 
and ilemonstrated through certain graduate-levef courses in the College of Education in which the majority of 
studetm chose to learn APL as a programming language for educatkmai purposes. Beywid the use of APL for 
instructional simulations, however, there appears to be reasonable doubt as to its feasibility as a general CAI 
language. In the experiences at FSU. little instructional programming with APL was perfomied other than 
simulation (juckages. 



Nattir^ Larmaage Prmessing 

One of the initial dreams for research in CAI was the possibility of nearly natural communication between 
student and computer for the purpose of ioitniction. Over the past few year^ this research has become actuality 
l»asi!d t«j the concept of semantic processing and semantic rats as defined first by Oiillian i\9m\. Carboneile 
(19701 terms most CAI systems "ad hoc. frame-oriented" systems, in that all stimuli presentations and r^ponses 
most be fully planned ahead. With a natural language processing iy>tems and appropriate semantic-oriented data 
bases, such preplanning is not necessary, and, in fact, the flexibility of the system may be moiB.powerful and 
realistic for learning. 



FSU Oiatogue 



The thrust of FSU natural lan^ge research has b^n not ^ luch on the technical aspects of computing 
that must be solved for such a system, but rather on the pedagogical considerations for specifying and pfenning 
the desired type of instructional dialo^e. To accomplish this, a systematic apiwoach was developed which 
utilized the typicaHy stated steps fcr instructiwial development, oriented toward natural langua^ instructional 
dialogues. More specifically, the analysis of instruction was based on determining a semantic network of infor- 
mation which the student was expected to have learned by the aid of the dialo^e. On the basis of this seman- 
tic network, a data base and questions to the student can be ^lecified as performance objecti\«s and so 
programmed. While this particular type, of system is limited in its power for cteveloping semantic netwwks. 
particularty for syntactic analysis oi student input, the project was nevertheless a surxess as a prototype develop- 
ment. A complete dialos^e was developed for a humanities t»urse, and students using the dialogue in this 
course did generally acquire the semantic content as specified in the systematic ap|»-oach to the dialogue 
development. A more extensive research po^am would be required to fully research and evaluate the technique 
used 

Future 

J It is anticipated that further exploration to the area of inter«:tive systems is very bri^t. Present langua^ 
capabilities will continue to be expanded, while more interactive dialogue and natural languages will be developed. 

Irtstfuctfonai Mana^rmnt'Syilems 



Instructional Management Systems consists of the computing software and hardware necessary for 
control and monitoring of the instructional f^-ocess- Operationally, it is cxwivenient to talk of the instruction^ 
data base as the asscxriated files within the data base as well as the manipulaticKi of those tiaU for control and 
monitor functions. It is prime tenet of computer systems analyses that the specif fcation, design, and utility of 
instructional systems data bases are tied to appropriate identification and integration to the educatfcHial 
functicins. The rest of the FSU approach has been development and dissemination of data base Software appro- 
priate to educational users in both service and research. 
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APL File System Package 



One of the prime {imitations of the APL/tSCK) svstem. and inctec^J most APt systems, was the Jadt of d 
file estabtishftient and manipulation capabitity. To overcome this timttaticm* the APL/ISKI File-Attess Sub- 
ruutme Pacfca^ (SM10| was develcjpeil. The packa^ p^mUs offline act^^s to data generated thr<Ki^ instruc- 
tional and research appticat'oos of ^ system, tt was found that needs of usm whit^ must be lilted by the 
system are: (a) the capabiHty to acce^ data for analysis by other computer systems^ (b) the capablfity to 
access data for acc^iring hard copy^ in a quidc^ nKide than is avatf^bte on an iBM tSCK) system, (c) the 
capabfUly to build input data to APt fites, and Cd| the capability to extend data b^e saz^ It should be noted 
that the fite access subroutiTO package was built ^ a direct extension onto the file system capability which 
air^idy had extended the APL data base functions {SMB}. For example* these capabHities allowed a nn^jor 
project of instructional usa^ in information retrieval systems for the Schocrf of tttirary Science. The School 
v^s able to devofop a system of automated abstr^rng and retrieval <ki the APL fite system. iTtte system also 
lirovid^ instruction for graduate library ^<en€^ stud^s in the areas of autoimited library re^rfewl systems^ 
Experience with this system indicates that any system of computers in education must provide the capability 
for a well defined file system with variable structuring and flexible maniputation. 

Data £ita^ Msniputation cm the Caar^writer Systan 

^lectffc requirements were found for manipulation of lart^ files on a regular basis. Because of the volunmes 
of data collated in CAl r^earch, hi^ly efficient proyamming was needed to allow maximtatlon of fK'ogram 
execution times and therefore operations. One of the major problems in all compute systems^ not only in 
education systems, is the bottleneck of input and output functions. That is, proyam exetattion is no faster than 
tJie time it takes to pass data throt^h it, the "thr(Xjgh|Kit" time. Howeva^, throughput h a function of the 
time for input and output, also. Throt^hput improvements vi^re act^lerated by progp^mming internal execu- 
tion speeds so that pro^am$ api^oached the l/Q boundary conditions vsfhHe still m^imizing exet^lon efficiency 

Sort and 

An internal, sequence-sensitive, bi directional sort was developed to take advanta^ of the fact that CAl 
records and data are not raidomly ordered, but are ratfier more often sli^tly out of sequent. This cxxtirs 
because data are recorded fcMF students within specific a>urse$ at approximately the wme time and dates. The 
bi-directional sort locates the correct sequence position of a currently enwuntered, out-of-s^iuen^ record in 
no mora than ei^^t comparisons for any IIX) resp<mse records. The software mand^ment alk^ t^ocking of 
records in ^oups of In additic»i to the fast location finding of the sort, no data are immediateiy swaf^^ 
because a table of locations is kept and accc?ssed for addressing the ifKlividuaf records. That is, intermediate 
data swaps have been eliminated by indirect data examinations via a bi-directional Hnked-index listing procedure. 

In addition to the high speed sort, tt was also necessary to develop specialize nrier^f>g jKograms whidi 
allowed the ortierjng of data bv specific request. For example, it might be dirable to have records only for a 
given course, records for a given course within a specific time period, or records only for a se^®it of a course. 
For computational efficacy, it is expedient to block these specific records toother and eliminate other. The 
mer^ and sort programs, w^ien si^ificaHy oriented toward the type of semiordered data accumulated in CAl, 
fulhtl these requirements most eff iaently. 

Instructional Support System (ISS) 

A major softvwre development project undertaken at FSU was the redesign of the FSU data management 
system and IBM CAl system such that they operated under one complete support and ofrerating system. That 
IS, either system could be controlled under one set of operating procedure. This integration of the two 
programming systems allowed ^eater optimisation of computing hardware utilization, and a significant improve 
rnunt m computer operations. 
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M^j^S^mnt Qp&ratiarmf /^acedur^ 

Prim*? sources of maximizing the utility of any imcra€tkir<al imnagtsn^nt system are Xh0 op^ations 
Pfoceilures dfui scheiiuling. FSU has amiie 4 maior effort tu taisurir that the human component of the 
instruc{i<ma» nwna^ment system is wi?H integrated with the software and hardi^ai^ components. &>ftW2»^e 
pro-ams which allow editing and preparation of data for data analysis were develop^, a]<Higwtth com^ 
plete user documentation. For researchers, the special f^ed for maniputattfHl data for use in statistical soft- 
^re packages was taken into amount. Given the variety of data which may be collected on a sophisticaied 
CAI or CMl system and tfie varieties of research designs for analysis requirements^ flexible and powerful 
control of data must be available for res^rchers. 

fuwre I 

In the future, investk^tions sfUHild key in on the envelopment of new ^hnoif^ic^ for recording 
amounts of data during computer-aided and computer mana^ instruction activities. As instruction becomes 
more individuafi/ed and personalizmt^ rwiwging ^d monitoring systems must be expwncted. 

Data Analysis Systems 

The business of alt instructional research is analysis and interpretaticMi of data gathered. In order for the 
data to be Mcpful, however, the CAI/CMI system must be well integrated with data analysis systems, Tfic 
thrust of the FSU effort with concern to data arialysis has been in tlw direction of provrdmg new user 
cai^biiities for analysis of data. 

Otfttine versus Offiitw Sysi&ns 

Many CAI researchers have sug^sted the need for data reduction and {[^^ntation in an onlif^ mode, 
with data output to the researcher as students proce^ throcK^ imtructional treatments on dJi^lay ctevii^s. 
The offline mode allows gathering of data on ^ferm^iate storage media which are ^rcessifole at later times 
for data redt^rtion and presentation. The offline mode has been primarily emphasized at FSU. The advanta^ 
of offline data recording are in ease of operations and flexibility with which raw data can then be trans- 
mitted to data analysis systems. Many manual operations necessary for online <kta gathering are bypassed with 
offline functions, 

R^ort Gen&ration 

Both the EDIT and DATA PREPARATION fxo^ams M^ovide punched output of data collected c^ing 
CAI coursewriter activities for use on various romputer system statistical piKA;as^. In addition, two proyams 
were developed at FSU which provide summaries in the form of j^-intec^ output report of CAI data. The ITEM 
ANALYSIS SUMMARY pro-am provides summary statistics sudi as the fwcentage of students who answer^ 
an item correctly and the average latwcy time for tfie item. The DETAILED PRINT pro-am \>U(\X% a com- 
prehensive ctetailed description of a student ur subject's rerord. Information such as cwrse r^me, data, EPID, 
MID, student response, response latency arnl value of counters and switches is provided. This fw^o^am has a 
great deal of flexibility throu^ the use of 32 cations foi output specification to select or suppress sf^ciftc 
information elements. 

Future 

If, In the future, the trend continues toward more powerful general purpose ami algorithmic- type languages 
for CAI, then more general and powerful report generation systems will have to be developed with some loss in 
execution ofitimi/ation for the qain of power and flexibility. 
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A MODEL FOR THE USE OF ACHIEVEMENT DATA 
IN AN INSTRUCTIONAL SYSTEM 

Rofcert L. Brennan 
Department of Education 
SUNY at Stony Brook 

INTROOUCTIOM 

Any attempt to discuss practical dvp^'CK; or evaitiatton m instructiofiai environmmts is apt to suf^ from 
erthef one of two shortcoming: the discus^iion may be so fjeneral that it fails to itrovicfe fiHich ^d^nce to the 
evaluator in the ttecision-making fH^ocess, w the ch^^cusston may be so ^cif icatly related to a parlicutar mode of 
instruction that it lacks generalizabititv to iM^r modes of instruction. White recognizing thes« two pitfath, we 
nevertheless subscribe to the tielief that few things are more f^^acticat than a ^rod ^eoreticai model provided that 
the model has a reasonably aroa^f ran^ of applicatntity. 

Thus, in thi% paper, we propose a definition of an instructional system v^idt. although not alt-inclusive, 
does encompass many frequently use<l mi>des of instruction, Also^ we specify a rmKfel for considering the kinds of 
achievement dai.i and time data ihat should be coltected, why such data stioutd be collected, and how tfiey should 
be used. In ^lort, this paper presents an mte^atinn of some useful theoieticat concepts witfi a constderaticm of 
many oracticai prohiems of decision-makiitg m the context of an tnstri^iionai system. 

As tndicnt'Ml m the title, we restrict ourselves here to a consideration of achievement data and time data for 
evakf.rinq the cognitive asf^ts of an instrut^tional system. Given ttie current $tate<of-the-art, cs\e mi#it argue that 
achievement data and time data often provide the most useful and interpretable tnformatton with regard to 
decision making in an instructional system net^itates the o^llection ami use of other types of data, as well. 

DEFliMfTiON OF AN tNSTRtiCTtONAi SYSTEM 

Let us def nu^ instructional system as a cteliberate, systematH:, and replicable orgmi/ation of a set of 
r<?soUf ce^ for the principal purpose of efffHrting student cMrhievement of clearly stated instructional defectives. 

This definition is somewhat restrictive in tfiat not all iedrning occurs as a result of cteliberate instruction, not 
all instruction is replicable, and not alt instruction is principaMy intended tn produce specific goals. For example, 
most discussions, lf»ctures. and seminars arr not repiicatTle, even thou^ specific instructional ^^s may be stated. 
Thus such modes of instruction would not usualty qualify as an instructional system, given the above definititm. 

Nevertheless, the above definition is ft*^n^»f al enoug^i to encompass most applications of c^oupi^aced, self- 
paced, and imlividuali^ed non-traditional instruction including, for example, most instruction of a computer- 
assisted, progfdmmed, or multi mt?dii nature Note specially that an instructional system, as defined here, may or 
may not t>e individualized. Note also that the tt?rm "replicable" is not synonymous with "inflexible"; irwleed, 
these two terms in the context of most instructional systems are incompatible- We argue that an instructional 
system should be replicablr m order to provuJf^ .i t>asis for imniementing objective procedures for improving the 
system, but improvement implies the necessity for some kinii of change, not inflexibility. Finally, note that 

1 Paper pr8S»otef1 at the Annual M^ttnp of fhf American fducutional Bassarch Ancciation, N#w Oflesf^f^ February 1973. 
ThfS astpvr was Drl^lnaliy onti tieci "Tb« Uw of Jetf Oista in NonT r/xiit <onal ln»tfuct*on/' 

^Th© rosoarch roiiOrt&n h<Tp»ti was par^itHlv «»uoi»orf«d by g^.int ^ IrnrT* the United Sl^ffK Office of €iiu£:afion. 2 2B1 18, 

and tf»« RM0»rch fotindifian of t#>«^ State Un»vt^r<iirv -if Now Vork, No 0477 03 031 71. 
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students are an importiint comfKHieot of an mstructiorMi system. In fact, w |:^apose here that students bs con- 
sidered as much a part of an inslructionat system as the instructfon used to teadi them and the test item* used 
to evaluate them. Thus, the proems of evaluating ^ instructional system mvolves making decisions about test 
items, instrucriofi, ^k/ students. 

C»J£CTlVE-ReLATEO MODULES 

One reason that so much of the Hterature on ev^uaiing particular instructional ^sterns lacks generalizabilify 
to other tnstructJonat systems is that the unit of analysis for the purpose of collecting data and making decisions is 
apt to vary TOnsiderably from system to system; and. often ^nou^, the unit of analysis varies even within the 
same system* 

In some systems the unit of analysis is merely the amount of instruction that occurs in some specified titm 
period; in other systems the unit of analysis corresponds with the instrtwrtion for some group of objectives whidi 
are taught toj^ther m uime sequence for p^^pc^ieal reasons. In lK>th of these cases, the unit of analysis corresponds 
with djvious physical characteristics of the systen). and, therefore, the unit of analysis typically Involve a nun*er 
of diHerent instructional c*>i^ivcs. HcMever, ^ kinds of decisions that must matte in evali^ttng and revislr^ 
an instructional system necessitate a consideration of all of the data and insuuabn relating to each separate 
objective, no matter when the data are collected or wfiere the instruction cKcurs within the system. 

In short, the basic unit of analysis in an instrui^tonat system shmild be the ob^ve* In order to emprfiasize 
this fact and facilitate the collection and analysis of data for decision making, it is theoretically and practically 
useful to view an instructional system as consisting of a discrete nimiber of (dftectit^^i^mt nHH/utes, As employed 
here, the phrase ''objective-related oKKlule'' refers to all of those factors in an instructional system thai are directly 
related to a particular instructional ob^tive. Note especially that die term "mcKlule" is r^t used f^re as a 
descriptive characteristic of the f^ysica! layout of an Instructional system. The central aspects of an objective- 
related mcMluie are the objective itself arKi ttie instruction intended to tea<ii tfie obj^ttve. In addition, an objective- 
related module contains all of the data directly relevant to ttw parttc^iar ob^cttve. 

At fkst this conception of an instructional system in terms of ciijective-related module$^ may s^peiBt too 
theoretical w too trivial. Yet, for purposes of evaluation, the concept of an objective related modular has sevmai 
advantages over many other ways to outline or describe an instructional system. First, and most importantly, 
thts concept directly implies that the objective is *e tiasic unit of analysis in an instoictional system. Sea>nd, the 
objective-related module conc^t emphasizes the relationship t^tween tl^ objective, instruction, and data. Third, 
any instructional system can described in terms of c^jective-reiated modules, regardless of how the instruction 
1$ sequenced or packa^. Fourth, tt>e objective-related module com^pt greatly facilitates an understanding of 
many of the issues and problems surrounding the eolation and use of data in instructkmal $ysten». 

A TAXONOMY OF TEST DATA AND TIM€ DATA 

FOR EVALUATING ACHIEVEMENT IN AN INSTRUCTIONAL SYSTEM 

* 

Any discussion of data immediately raises two quest imis: for what purpose should sudi data be collected 
and what kind of data should be collected? Here we restrict the sa>pe of these two questions to the domain of 
evaluating cognitive achievement in an instructional system. 

Purposes of Clata CcMlection 

It) general, of course, one can say that data is collected in the environment of an instructKHial system for the 
purpose of evaluation, where, according to Stufflebeam (1971 ) "evaluation is the process of delirwating, obtain- 
ing, and providing useful information for judging decision alternatives {p. 267)". 
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More specificatly, one cxjuld s3y that data ^tioutd tHs coHected for the pi^lKi^ Of dfa^pmtic, formative, and 
summativt* aMtudtton {Btoom, Hastmgs, and M^aus, 1971 1. If me consicters evaluatitm as a ctedsion-making 
t^-OGeis. then the diacyno&tic forinutive summative trichotomy refers prifmrify to deci$kNl-making functions of 
evatudlion. How^'ver, we pi efcr to emphasf^ that the pur|K>sc of coilecting c^ta in the envirmm^t of an 
instructional system ts tp niako decisions wtth regard to specific a^iects of the instrtKlkmaf system, rtameiy : 
la If instructioin (bj stuctents, and (c) test items. That is, we i»^efer to emf:^iie the cAject of tfie decision- 
making pro<^s seems to identify more cleariy the specific nature of the decisions that typtcaily need to t% matte 
in an <^going instructional system. 

Decisions about instruction are usuaify of primary importance; t.e^ cme wants to assess the effects off 
instruction especialiy for the purpose of identifying instruction that rm)uires revision. Sut^ decisions ere often 
viewed ^ part of the process of formative evatuation. In ofcter to make ctecisbns corH:emif^ v^ther or not 
instruction shouW be roviseil. we will argue ^i^re that data should Obtained whidi can be \^ to determine 
instriictfona^ effectiveness, efficit^icy, and retention. 

Ctec«sions about students typically mtAude decisions rancerning stiKtent r^iecenmit ami c^ifi^tion. Such 
decisions are often viewed as part of the i^ocesses of diagnc^tic and summati^ e¥ali^ti<«n, f^>ectively« 

Decisions concerning test items also need to l» made In irtttructic^^ systen^ ^>ecif k:atiy« wie needs to 
determine the reliability and validity of tests used as part of tf^ instructional syst^. 

Tyim of Data 

In order to make the kinds of decisions indicated above, two gsn0n\ types of data are useft^: ai^ievement 
test data time data: instructional time and retention time. ASw, we will consiiter tvw gmm^ typ^ of 
achievement te^rc criterion-referenced tests and nornv referenced tests. As Gt^er (1^3) notes: 

The scores obtained from an achreven^t test f^ovicte i^im^lly two kinds of infonratkKi. 
One is the degree to which the student hm attained criterion performance, for ex^frte, 
whether he can satisfactorily fwepare and exf^imeittiU report^ or solve c^ain kinds of work 
problems in arithmetic. The second kind of information that m ^ttei^mmt test score 
provfiles is the relative ordering of individuals with rttpect to their test perfc^mam»« fen* 
example, whether student A can solve his problems nwre cruidcly than stuctent B. The 
pfincipa\ difference betv^n theas two kinds of informatkMi lies in tfte standard us^ as 
3 reference. What can be called criterion-referenced measures depend ufxm d9SOhfte 
standard (p. 2). 

In other words, the results of an achievement t^ can be used to rmk studmts aic^g a relative continuum 
of capabiJity with respevi to a specified group of p^sons Inorm-referencing) along an d:^ute continuum of 
proficiency with respect to a specified t^haviord criterion {ar^erion-refereiicir^K As Nitko 119/1) indicate, 
one can make a distinction between a criterk>n-reference test % defir^ by Glaser ^ni Klaus (1962^ Glaser 
(I963f, and Glaser and Nitko 0971) and a mastery test ^ctefined by Bloom (I^^K Here we will not make any 
distinction between these two tests since, in the model we will discuss, any sudh cfistir^ion seems to have little 
practical stghiffcance.* 

In particular, we will argiie here that one can identify ei^t different types of data for an ob|ective-reiated 
module that provitk3 meaningful sources of information for decision-making. These types of data, listed in the 
order \n which they would usually be (djtained, are as follows: 

(a) Pferequisito test data, which indicates w*Jether or not a stui^t h^ the b^grcHind diaracteristics 
(dttaintnimt of r>rf« vKHjs ohitfcttves, aptitude, etc.) thouf^ to be necessary in order to achieve the 
obj*?ct<ve for the mcHlule, 
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{h) Pr**ti*iit tiaiu. wh»di im;asures a stWlent's per lormanci? on the c^^ctive ^iot to imtructfon; 

<c) Instructional time, whidi <$ the length of time a student si^ends under^ng tmiructi on far the 
oiifective; 

(ilj Cntenon referenced fiosctest data, which measures a student's f:«rfonnaiice on the objective 
uiimedtatt'iv lifter <n!itrycl{un; 

M Norm fef«?reiic^ posttest data, whidi is coHected immediaitety foliowtng instruction and measun^s 
student performance relative to the performance of other simitar students; 

(f ) Retention time, vwhich is the length of time intervening between the posttest iusi^Uy criterion- 
referencetf ^ and a subsei|uent retention test {usually criterion-referenced^ 

Criterion-referenced retention tast data, whidb is collected some time after instruction and measures 
student performance on the G^iective for the module; and 

<h \ Norm-referenced retention test data, v^idi is cotfected some tinw after instrcKrticm and n^asures 
vtiuient p^'rforn^nce refati^ to the performance of other similar stuctents. 

It IS of tt»n assumed that only critericm referenced or rmistery test data provide mewmgful informaticm for 
evdtuaftuii decisions with regard to instructional systems. We agree that criterion referenced data is mom important 
than norm referenced data in the ctntent of an instructional system; however, we ar^ below that norm referenced 
data sometimes provuie useful additional information for decisioi-making* 

A Table for Relating Data Type and Use 



These data for an objective-related module are ^fisplayed in Table 1 whitrfi, in Edition, indicates tftwe tyfwrs 
of data that are of primary importance for making particular kincte of decisions with regard to instruction, students, 
and test items. In essence, TaWe 1 provides a kind of taxonomy of adiievement and time data that are u^ful in 
evaluatmg instructional systems. 

TABLE 1 

Taxonomy of Ai^ievem^t C^ta iml Time Data for Oecfa^oivMbking in m Instnmkmd Syt tsm 



DATA USED TO 
MAKE DECISIONS 
WITH REGAI90 TO: 



Instruction: 

Short term effectit/eness 
Lony-tHrm effectiveness 
Efficiency 
detention 

Stuiients: 

Placement 
Certification 

Mastery 

Grd<hng 

Tests: 

Validity 
Reliability 



DATA FROM AN O0J£CTIVE-ReL.ArED MODULE^ 

REO. PRE' INST. POST- POST- RET, RET. 
TEST TEST TIME TEST TEST TIME T^ST 



n,rs 

RET. 
TEST 



# • • 



Tti« dut*f fire UiWii f r<Mi l4>fi to r^ght in tfie chrcmologir^l orrtor that tfi«v would u«M»l!y t« irollected. Afit^risKs indlcAte th« 
prtnclpal kinds of (iaXB appropriate for parttculdr di>c^«ian«. 



**Cr«t«riOfi refererKOfJ r9t«ntion tersL 
^NOffti f€?f«?rffncff<,l fet<?nt<on tost. 
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It is, of course^ quite possible that «i partitutar obi^ive retated imxiirfe imy not contairt of tht» data 
fTidica.^ 4fi labie I. It also quite likely that, in particular oh^^^-reia^ modute. a tett msy, in fact, co^ii fst 
of only one item. These and other pmsibilittes ami tl^ir implicattons will be b^0dldd later* For the present f*fw, 
let us assume tlial, tor a |Mf ttcular obiective-related module, ait the indicated date are collected and each of the 
te^ts confatii a number of eqirivafent items fw measuring ttie same objective. In ihe foitmving sections of this 
paper we will consider in more detail, the nature of these data, they should be collected^ and ho^ they mi0n 
be First we consider, in some detail, the use of data for evaluating instruction and sirtisequerUy we provide 
a less extensive consideration cf the use of data for evaluating stucfonts and test tteitts. 

EVALUATING INSTRUCfiOK 

As Crc^bai^ C1963> irHlicatt)d, when evalt^ticm is performed in wcter to effect the lnwo««»«fi*»f an 
instructional system, the chief '^m is to ascertain dian^s the system prcKiuces in sttid;!itts in order to identify 
efemems of the Instructional system that require revision. 

One of the most theor^iicdly appealif^ i^ocodures i(tentifying instructi<M^ ttiat reiiuires miston 
involves determining departufes of attained results for the idei:!. That is, the {»'Ocess of evaluating instructron may 
Ixj v«ewijd as the process of comparing the actual effects of instruction with the effe^ that would be produced 
in an instrtictional system as one v^rfiich would nesult in students mdiieving a niaxiinum level of performance 
pn all (^jectives m the shortest possible time and moreover, stuctonts should maintain this level of performance 
for 31 long as necessary. 

A Ratitmale for Dwision-Mafcinf 

In a sense, the characteristics of an ideal instructional system provide ap upper bound for ob^ervet.^ 
measures of effectiveness, efficiency, and retention. In a similar f^Jon^ cme can idontsfy the characteristics of 
a mfnim^ty Bcc^tst^e /nsiructiOMt syswm and, thus, provide a kind of lower bond or base line for these 
measures. If the observed measures fall within thet^ two tmumiaries, then the evaluator wilt not usually suggest 
the net^ for revision; if not, revision is required. 

The above rationale is quit^ simf^e to state, hoover, time are x\w major prcAlems in implementing the 
P'ocedure. First, one must si^ify the cut-offs that tfefine the lower bound or b^ line for the minimally 
acceptable instructi^al system. Second, if nevision is indiiated, then one must determine what needs to be 
revised and hc^ such revision should be accomplished. These two proMems have been mldres^ to sonri^ extent 
by Emrick 0971 1 and Brennan and Stolurow 0971 K resp«nively. However, tin ^utic^ to toth problems, 
given the current state-of-the-art, must rely primarily upon the combir>ed sub^ive (ucHptimt of the evaluator 
and the subtect matter expert. 

Decisions concerning cut-off are especially complicated. To the extent that an instructional system is not 
ideal, there then exists the possibility for improvemmt «id perhaps, the necessity for revisicMi. V^ether or not 
revisim is actually appropriate depends upon the cut off vati^ for instructitmal eff^ivan^, efficiency, and 
retention. The cut-off values, in turn, riapend upon the evaluator's and subject rratter spwialists's omibined judg- 
ment with re^rd to a>st f^ors, revision time considerations, and a number of o^er more sub^tive utilittes. In 
addition, iftese fi^tors often imply the need for different cut-off valii^ for different <rtijectiva-rela«fd modules. 

Yet even with these problems, the e^ove rationale does provide a very useful framework for identifying 
those a^^s of an instructional system that require revtsicm. In ai^ition, once cut-offs ate specified, the 
revision required/no revision required decision is essentially strat^tforward. 

Finally, even in the problems discuss«i above have one r^eeming feature - they emph^ize tfie necessity 
for a close working relationship between the evali^tor and the subject matter expert. 
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iminfctk>nd Effectiveness 



Table 1 itidicates that inslructrunal effectiveness implies a t mi^tisonof ^oficieney measured at two 
fitiints in fmt* pru)r to instruction an at some time fa<towjng frntructtmi. can identify cither ^c^t*term or 
roncj'term eff ectiveitess depeniiing upon vi^ther m\e obtains cfata from a post^t administered imn^iatety after 
instrucitioii or from a retention test administered at some futiire time. 

The necessity for determining dtort-term instructional effectivem^ h readUy reco^i^; hcwever, the rwed 
to lit ternHtie long-term instructional effectiveness is ^fasfly owrloc^ed. Consider the siti^tion hi which a stmtem 
fails to achieve jn objective as a result of experiendr^ instruction for that specif M: ob^tive - the student c»n 
H<iwt^vi;r, eventually achieve the cA>|ective as a result of subi^qiient instruction. In this case, the learning that 
occurred will be ret tectKl m long-term instructional effectiveness but not in i^hnrt-tiNnn Instructional effrHniveness. 

It is important lo note thdt effectiveness cannot be ctetermined from a rr^surement taken at cwie point In 
time unless one can validly assunw tht (x>mptete ab^nce of f^^oficte.^cy f»^ior to instruction. Tills assumption 
should net te taken fi^tly , It is often (perhaps usually! the case ttwit at feast some students, f^^lor to instructiim, 
have Ipvel of pr of •Vnency er^ate-^ than zero. If this fact is overlooked, estin^tes of instructicMtat effectiveness will 
Of* infldtect 

It IS also important to note that any comparistm of pretest and postt^t, or pretest and retention test, 
performance is meaningless unless the two tests measure the same <Ajef^ive at the same level of difficulty. One 
coMiTtun vtoiatton of this rule occurs when the f^etest is an easier test than the posttest or retention test. This Is 
a circ*j;nstanc^ that vvi!l result in a ^flated estimate of Instructional effc^tiver^s, 

f mtructkm Efficteficy smi Retentkm 

Histructional efficiency and letentton involve a oHisideration of tK)th t^ data and time ifete. As indicated 
in thk} defjrtilion of an ideal t nstructicmal system, data with regard to instructional time and retention tinw are 
very im^wrtant in assessing the extent to which a given instructional system deviates frcmi the icteal. 

Table 1 emphasized that instructional efficiency involves a ^int consideratKHi of ^ort-term instructional 
efhfctiveness and instructional t'?rc, instnictmn^! time fot ^ yivtjn ob^fcttve may ctmstant for aft student, if 
instruction <s grr»i»j^p2OTd; nr. !n*:tructionar time rr^y vary for each Student, if instructional effectiveness and 
matnictional efficiency are functionally related, differing only by a constant tinw fiK^tor for all students. In the 
taltei c^ye, instrm^huiuii effectiveness and efficiency may provicte my different jnforn^tion. For ex^ple, 
msf ruction may be relatively effective fcK an entire gr<nip of students, tKft s?me of liw sttnfents may S|:^nd so 
much time in instruction that tnstruilional eff idency for tfwm is exceedingly low. One mi^t even theoretically 
argue that a high level of instructional effectiveness can be guaranteed if eff iciefK:y is of no importmce <see 
Car oil, 1963). It should be noted, however, that neglecting to consider effidency makes little sense unless one 
places a very low premium on student time, for most instructional systmis, stuctent time is a very valuable 
commodity that wie cannot afford to squander; hence, instructional efficiency assumes consictercMe importam^, 
t^sperid'ly in self paced or individualized instructional system. 

Efficiency as described above, is, more orecisely, ^ort-term inMn«*tionat efficiency: One might argue for 
consujeration of the tinalogous concent, long-term instructional efficiency, however, to dn so would involve 
ronUjsing instructiory?! time with retention time. 

In this context, retention time may be defimru as thi> nme intervening berween a criterion-referenced 
fx>Mtest and a subsequent criterioii-feferenced feieittion test. Also, recall that in our terminology, these two tests 
ate tests of a single objective since they are part of an obtective related module. Thus, as indicated in Table 1, 
retention initcates the extent to whidi proficiency (cm an obiecttve test^ immediately follcwving instruction) is 
m^»'^tdinfci ♦p' ? :>:»'ticutar period of time. In short, retentic^ takes into account three factors: posttest per- 
formar^ce. retention test performance, and retention time. 
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Note that, when vieweil m the context of a» ob{€2Clive* related module, it is clear that reti^tion timely 
vary for each objective; and, moreover, in a self-paced or indtvtdu«ili2ed tnstructiofidl syAem retention time for 
each obiective may vary for each student. These variations in retention ume are often overftmked when a sin^e 
retention test is useii as a mt?asut*« of |H?rfnrmanrt? for several obiectiv^. each of wfwch was tat^t at a different 
point in time. This is one practical reason for coitecting and analyzing data in the ccmtex^ of dbjective-reiated 
modules. 

TTfe KteasurenM^ of Change or Gain 

^fiis far, we have heen careful to refer to instruction ef fectivene^, efficiency, and retention cmly in tenns 
ol the tiata and concepts that provide a basis for defining these terms. From tfiese omsiderdticKit it seems dear 
that any actual measure of effectiveness will be in essence, a measure of diange dr gain and any actual measure of 
efficiency or retention will incorporate a nteasure of diange or gain afc^g with a ccHisideration of instructional 
time or retention time, respectively, 

Many measyrt>s of change or gain have been reported m the literature. fS^, for exwipte, Harris, t9fi3; 
OuSois, 1962. and Tucker, Oamarin, and M^skk« 1986.) R^ntiy, Cronb^ and Furfoy (1970) ha^ argued 
that ''investigators who ask questions regarding gain scores wcHiki c^dinarily be better advised to ask tfteir questions 
in other wavs (p.80)/' Unfortunately, this statemeni has been indiscriminately interpreted by many as a blmiket 
indictment of the measurement of d>ange m educational and psychological rcseandi; yet, Cronbadt (I^^K 
self, pointed out ten years ago that "when evaluation is carried out in the service of course improvemf^nf ^ the 
chief aim is to ascertain what effects the course has - that is what chaages it produces in pupils (italics ours, 
p. 675)/' 

Hie two statements by Cronbach m the f:»^evjous paragrafrf) are not nece^^riiy ccmtradictory. Most mea- 
sures of chanj^ reported in the literature have arisen out of a correlational model that assumes an interval scale 
of measurement: yet, in practice, tr^itioaal psychometric measures fail ^produce an interval scale* This is a 
princfpal ru^s^uti why Cronbach and Furfoy (1970) argue that the usual measures of change are often suspect. 
However, this objection ceases to exist when one has an absolute measurement scale tiiat m^sures interval 
scale units of some entity (Carver, 1971 ). Now, one can stron^y argue tfiat the criterion-referenced tests used to 
evaluate instruction, as descrii>ed above, shouid, in fact result m absolute measurement s<;ates« since ea^i test 
con^ibts of equivalent items for measuring performance on a sin^ ofajectiw. Thus^ in order to assess ir^ructionat 
effectiveness, efficiency and retention, one c^an^ disct^s^ in the literature. In our opinii^ however, many of 
these measures of change have f imitod utility for evaluating instruction in the tfiat tfiey imialty fait to 
ikovide the evaluator and subject matter specialist with the kind of easily interf^t^ile information that 
facilitates the ictentif ication of instruction that requires revistun. 

Also, in some cases, measures of change reported in the literature are closely related to hypothesis testing 
in the context of cfassiceil inferential statistics. As Lumscteine (1965) points out; 

A weakness of the statistical habits associated with before-after and gain exf^imants is 
that the statistical tests employed are ^dressed to hypothesis testing rather than to 
estimation. . . it is truf? that in determining tlie effects (of and instructional system) one 
wants to rule out tite null hypothesis that obseived gains can be dism^^ ss a cham^ results 
results; i.e., one wants to show that effects produced were statistically reliable. How- 
ever, what is obviousiy of more interest is a good estimate of the six of the ^in. . . (p. 299). 

l^erely showing reliable evidence Uyi some ^in is of little practical important^ in evaluating instruction, because 
the existence of some unspecified amount of gain does not provide a meaningful basis for chiding whetfter or 
not revision is requirf*fi. 
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Another limitation of many classical statistical techniques ii evatuattng instructicwi is the frequent a^mp*- 
lion that stu<knts be rjudonily s»it«pKHr from sotTw s^ctf ml |:Kiputati(»i. Often enouyh, students in an instruc* 
tional system cki not constitute i$ random sample; at best they constitute a representative sample of the kinds of 
stutlents for whom the instructional system is judged tc be ai^opriate. 

In short, io theory and in practice, many dassia t inferentiat statistical techniques fat! to provide ttw 
kind of intenw^etable informatimi an evaltwjtor needs in ord^ to identify asf^s of m instructional system that 
require revision. Thus evaluators often rely hi^vily upon descriptive statistics st«*i as difficulty levels, error 
rJift^s, percentage of maximum possible qain (McGuicKm and Peters, 1965; Brmman. 1970K etc,3 statistics 
moviiie a tiielhod for aggregating raw cfata into measures that have relatively $tral#itftorwarct interpretations. As 
such, descriptive statistics, wlwi used prudently, provide very ut^ful information tp the evaluator and the subject 
matter si^cialist Frequent ly« tire use of descriptive statistics is criticised betauie they fail to provide art ob^ctii^ 
ly iipftncd bases for infei ence to a specified peculation; however, » statisticians continually emfrft^tze, it is at 
best, misleading and sometimes meaningless to employ in^enttaf statistics wfien the uncterlyii^ assumptions aiB 
not fulfilled or w^f»n the resulting information is not inteipretaMe in the (Vision-making pfCK:ess. 

Thus, in our opinion, the utility of descriptive statistics, w4ien interpreted pri^ntiy, cdnnot to denied. 
However, it is often difficult to interpret simultaneously ^veral different, numerically reported ctescr iptive 
statistics for a particuter (^jective-r^iated module. One u!^l way to api^oach this problem is to employ 
^aphieal and tabular presentations of data such as tho^ indica^d in Figures la-Id and Table 2, respectively. 

Figures la-Id are examples of ^;S^ical tedinique that can km imd fm either of two p^irpos^ fa) to present 
descriptive statistics for one stuctent when pretest, postt^t, and retentiim test consist of a nun*wr of items, w 
(b) to present the average desaiptive statistics for a nundier of stiKients even wbm any or ail tests consist of only 
one item. In the fi^mr case, the vertical axis represents f^fomiance in terms of perc^t of Items cwrect; In the 
latter case, the vertical axis represents the avera^ performance for stiKlents. The hc^izcmtal axis represents ttie 
observed instructional time and retention time. The solid tine, depicting bast ine daiB, provide a kind of opera- 
tional definition of the effectiveness asi^^cts of a minimally acceptable imtructional system. That i^, the base Hne 
pretest scwe rei^esarfts the chance level of perfc^mance if students f&spund blindly; and the t>ase line fittest and 
retcfiitfon test scores represent cut-offs assigned tiy th« tivdi i0tor. 

In Figure la both short-term and long term effectiveness are kfeal In ^ner^, no revision is required on the 
basi^ ul these two measures so long as all points on the observed data line fall atove the faa^ data line. Ho^^ever, 
even in tfn? case represented by Figi^e la, revision mi^t be required on the basis of efficiency if the tvw-hour 
instructional time were iiKiged to be inordinately long. Furthermore, one cannot claim that retention is ideal 
unless the indicate ei^t week retention time period is a^opriate. 

In Figures Ib d revision is required on the basio of short-term efl^i^^ness and/or long-term effectiveness. 
Fic^re lb indicates a case in which instruction has no ot»ervabie cognitive effect. In FIguie 1c, revision is required 
on the basis of long term effectiverress. Figwe Id indicated that instruction for the ob^ive results in a reversal of 
learning which is corrected by subsequent instruction that is not directly related to the objective* 

Another descriptive t#M:hnique for vi<;ually displaying effectiveness data is given in Table 2. Once an evaluator 
specifies pass/fail cut offs^ for the pretest, posttest, and retention test, then each student falls into one and only 
one of eight mutually exclusive cate^nes. As indicated in Table 2, for students in Category I {fail pmest, pass 
fxisttest, pass »^t*>'iliontest) instruction has tx)th Icxig and short-term effectiveness; thus, Cate^ry I represents the 

Bfenfutn i 1969> for a rtfvtew of the Iftfifoture relottng lo doscrtptivo mttstics xh&t hs^ b«8n propofttd far u%& in 

f fhv le^t i.ontUts of a sinr|!f» It6m, Then p<isa/faU Is simply tronsJAteii as rorrw.X/wron^. 
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ideal. Each of thy uthtn ui IcMjaries iruitcute varying dey^ of lack of short-term and/or iong-term effectiveness. 
Category HI U>ass ^Ketest, pass postrest, pass retention if^t) i$ esf^lafty interesting in that it t^ovides an examf^e 
of the difference between evaluating instruction and evaluating students. Students who fall in Category Ul would 
usually beijvaluatetf fxisiUvely for the purposes of pimrement and ct tification. Howamr, for students who fall in 
Cjteyory HI, instruction has nut been effective; these sf uiients had atlatned the objective prior to instruction. 

TABLE 2 

A T^iar Display of C^iteHtm Ref^rmced 13^ 
for Assessif^ the Effecttvene^ of an Instruct^mal System 

SHORT-TCI^M EFFeCTiVENE^ 





ErrECTIVE 


NOT EFFECTIVE 


REVERSAL IN 
EFFECTIVENESS 




FAIL. PRETEST 


FAIL PRETEST 


PA^ PRETEST 


PASS PRETEST 




PASS POSTTEST 


FAIC POSTTEST 


PASS POSTTEST 


FAILPC^TTEST 


Pass 






" ■ ' ■ - 1 






retention 


Category 1 


Category II 


Category HI 


Category IV 




test 












Fail 












retention 


Category V 


Category VI 


Category VH 


Category VI H 




test 













\ 



t ntrt^ m t«^e ceMft of ttiis t9i>)e rafflr to tfie numbsr o1 stucfents in each c^ta^ry. Cote^^ri^f are muttisity 
»vi-iife*4,ti aiii^ e)ih«ustlv«; f,«.. each stucMfit laMt into one and onty one category. 



ComnHKits on O^ta for EiNiluatIng Instni^im 

The curtailment of data. In xbHb 1 eomtifersd ei^t dHferent idncb of data tfvat wetw varioiB functions 
in the evaluaticm of an instructional system. In a particular instructional system it nwy not be fe^ible to collect 
ail of these data for ail db^ctive-related nwiules. In sudi cases, the evaluator will not be able to ^ess at 1^ 
one of the four measures discussed above* This ^cAlem is perhaps inevitabte, but it need not be too S8rioi« 
ixovided that: {a) the evaluator plans the data collection privities beforeh^d so that the maximum amount of 
data (s collected for the most critical objective-related modules, md {b) the instructional system under^ a 
continual --cess Of replication, evarfuation, and revision. This is one reason for our insistence that an instructional 
:,vstem replicalHe. Even when the amount of data collected is som^hat curtailed, the ojntinual process of 
replication, evaluation, and revision <ihnfi|d result in a mnstantly improving instructional system, 

Prerequidtc test data. As noted in Table 1 , prerequisite test data does not provide direct evidence of the 
effects of instruction; however, once instruction is identified as requiring revision, stud^t performance on pre- 
requisite tests can provide valuable information in pinpointing the nature of specific diff MUjlties» For this re^n, 
the nature of prerequisite tests and their relation to specific objectives should be given careful consideration. 
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Testing additiami <^i^ti\ms, Cronbach (1363} claims that evaluation whose purpose is lo effect the improve- 
ment of an instructiotial system should ideally include: ''measures of all the types of f^ofidency that mi^t 
reasonably be desir^l in the area in i)uestion, not just the selected outcomes lo whicii the cuv riculum directs sub- 
stanltat attention (p. 679)/' 

Essentially this is an afflumem for the inclusion of tests measuring obiectives that have varying degrees of 
relationship to the actual objectives of the tnstructionaf system, Cronback ar ^es that such tests are necessary to 
assess the extent to which a particular instructional system is appropriate for users v^o have somewhat different 
educational goals. We agree« but it ^outd \m understood that student performance on such additional c*>}ectives 
IKOviiies evittence prinf>anty for asst^ssing the enter fiat vatidtty of the instructional system, not its internal vatidity. 
Thits, stuctent performance on such additional cA>jective&does not provide very useful evidence for revising the 
existing aspects of an mstructfonal system, although tnts evideno? may (^ovidea basis for adding effectives to 
4it*d/or deleting objectives from the system 

U%3 of nonn-ref&temed me^iHres, Unfortunately, it is often difficult to find or construct a criterion-referenced 
test to measure all the types of proficiency that ml#)t re^cm^ly be desired, as Cronback (1963^ siqigests. However, 
It is often relatively easy to obtain or construct a norm^referenoed test that can serve as a reasonably good measure 
of student performance on ob|ectives similar to that in tfHf mstructionai system, provided that tme analyzes the 
data very carefully. »n particular, the content standard test score techniqtms suggested by Bami (1962) may be used 
•vith norm referenced measures ami may provide useful additional information regarding the instructional system. 
Such data, for example, may pruvitle some b^sis for considering the app«^opr!ateitess of tfte effectives m the 
instructional system. After all, there is no a f^ion reason for assuming that the crib jectives in a particular instructional 
system aw the best objectives that could be chosen. In fact, as Ebd 0970) suggests, there is a danger that sm^ 
objectives ami their associated criterion-referenced test items, may be idiosyncratic- Criterion-referenced measure- 
ment must certainly carry the principal burden of evaluating instnictiw; however, to dismiss categorically the use 
of norm-referenced measures sees to be imprudent. 

EVALUATlftfG STUDENTS 

As indicated previously, the sttident is an important component of any instructional system, and, therefore, 
decisions about students are mtimately connected with ttte evaluation of the system. In fact, inuch of ttw literature 
that purports to treat the evaUiation of instructiofi, and the use of aiteritwi-referem^ tests m instruction, con- 
centrates primarily upon the evaluation nf students. Here, vw prefer to make a clear distin<^on between the evalua- 
tion of instruction and tlie evaluation of students. We rvote that decisions concerning ii^tryction are based upon an 

assessment changes, over tirrw, in student performance effected by the instonrtional i^tem. On the other hand, 
decisfons conCi^i »VffYg »iiid«;nis typicaiiy «nvol^ (^estions of student proficiency at a given point in time, as 
opposed to a comparison of proficiency at two points in time. In particular, two ^incipal kincte of decisions 
usually must be made when evaluating students: placement and certification. 

Deeisiom Concerning Placemmtt 

As indicated in Table 1, any one or any subset of the criterion-referenced tests in an ob|ective-relat**d 
nradule mi^t be used as a basts for different kinds of placenrtent decisions. For example, the prerequisite test and 
pretest are typically used as a basis for placement decisions with regard to the instruction in an oblective-retated 
module. However, these two tests usually play Jm feient roles in the placement prc»:ess. The prerequisite test is 
used to decide whethf?r or not a student has the knowled^ and other backgrCHind diaracteristics thou^t to be 
fw>^sary for attainment of the instructional obje'^»««>: ♦hat is, tne prerequisite test is used to ictentify students 
who will be allowed to experience instruction. On the other hand, the pretest, when employed for placentent 
decisions, is used to identify students who will be allo^.ved to skip instruction; that is, if a student achieves a 
sufficiently high level of performance on the pretest, the student is certified as having achieved the instructional 
objective and the student proceetls with instruction for some other objective. 
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Abo. th^ cf iterioft fefefinicec} |X5sttest attd, tess freqi^ntty, the critenoii-referenced retention test mav 
us«l for ptijccment decisions. The posltest often provides a basfe for (riacing 3 stuctent in ren^ial instruction for 
the objecttve in a particular objectrve-reiated modufe. or adv^cing the student to instruction for some other 
d^jective. On occasiiKi the retention test may serw the same ptacem^»m function as the (Attest, tiut mofe 
treuf **' »tlv the retention test, when usetl tor placement dechiions provides a basis tor placiny students in other 
instructionat systems. 

Eterisions Concerning Certlfieatfoo 

From a (iM^acttcai point of view, decisions on f^rtiftcatitm of student performance are often of two kirscfe: 
^ mastery or the certification of minimatly acceptable student performance and for wsKit of a better term, leading. 

Ma$WfY. As noted above^ the pretest may serve as a basis for certifying mastery or more precisely, ctetermin* 
mg minimal acceptable student performance. Similariy^ the criteripn-referenwd posttest or retention t^t may 
servH the sdnu? function. Any o* tests, ttwrefore, may serve ^ die basis for nuetery of a particular effective. 
Clearly, of course, the critical issue is: What ctmstitutes minimaMy acceptable stuttent performfliice tor tfie t^tfs) 
employed? In order to ansv\rer this question, #ie eva!uator must decide, first of aH, which test or tests will be 
used as the inslrument^s) for certifying m^tery. (Sometimes the posttest (or pretest) al<me may be used; pediai:^ 
more frequently, the retention test alone, or the retention test sHid the posttest pretest), will tje used* €H\oo 
Ihts decision is made, the evaf uator must specify a cutting score (or scores) for the ma<%tery decision* it should 
be noted that, tor the most part, the specification of cutting scores for certifying students can be made only on 
the basis of judgment on the part of the evaiuator ^d the subject matter si^cialist. However^ once su<^ cutting 
scores are specified, (fecision-theoretic techniques are avail^te for ctetermining wrfiether a stud^t's true score is 
above or below the cutting score (see Hambleton and Novidc, 1072). 

Grmiing, It is not our purpose here to either eondmie or condemn the use of grades and jading procedures. 
We merely observe that grades are often demanded by particular institutie^s, by parents, and by students. To the 
extent that grades must be rei^mied, we ar^ that the proc^ure for arriving at such ^ades stiould be reasonable 
and fair. Furthermore, we argue that the criteria for assr^ing ^ades shcKild be c<^sonant witii the use to which 
^^ades will be put. 

Few will argut that one way grades are used is to provide a rou^ estimate of whether or not a student has 
attained some particular level of knowled^ or proficiency. An instructional system, with its use of criterion- 
referenced tests, provtctes an excellent nwans for assessing such performance. It is also true, we believe, that 
cjp^ades are often i^sed as a basis for rank ordering students along a relative a>nttnuum of prohcierrcy; that is, 
^ades are often used to reflect an individual's f^rfrKmance with re^^t tn the peiformmice of other individuals 
in some sf»cified group. Essentially, this use of grad^ implies a need for norm-referenced measurenwnt. Now, 
^ Nttko (1970) and others have pointed out, criterion-refereni^d tests may be used to (^ovicte norm^referen^d 
kinds of information. Howver, criterion-referenced tests usuaiiy result in ne^ively skewed distributions of 
student scores that have little variability, and such distributions are not maximally useful for rank ordering 
students. Therefore, a test or tests specifically constructed to provide discriminations among student i.e., a 
norm-referenced tests, may often be desirable. 

Thus, to the extent that gractes are used to Indicate two different kinds of student performance, it smms 
rfiason;?N«» to argue that both cntenon referenced tests and norm-referenced tests are appropriate for determining 
grades. Ideally^ for grading purposes, criterion-referenced information and norm refererK^d information should be 
reported separately. However, if one grade must be reported, we believe that criterion-refereni^d information 
should \ye the more important determir^r of such a gradp. 

A By product of Evafuating Students 

A« Sornnson nn7l) noTirs, practt<:any any instructional system will work for some students and not for 
ottiets. \i\ order to take this fact into account, one of the developmental t^ks in definir^g an instructional system 
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involves the 5>|H?c*f4ca!*un uf ^tiidf^nis tur whutu ihe insti ucitun is intended. Even sa* when one evatiates student 
pedormance, it is oHtm fn^ssthte to identify the characteristics of students for whom the ifistructionai system is 
not workinc}. Tins infoimatum tit.iy pFiivkU! a Lwsis for revising the system or, in some ci^^ this information may 
be MSttd as a fjasis tin ir»s|i*»<:dvt. i Xh*' tvi^'s of stiidenfs for whcNU the instruction is appropriate* 



RecdII that in the u^nti^xt of un ^tlifective-telated module, a test is an instrument for measuring perfmmance 
Oft a smgle objectiw. Such a ti»s! ;deaf fy consists «f many items but could consist of wly cme item. Thmefore, on 
wcasicirn the issue of MvaUidtmif a t<!st nuiy resolvt! into the isst^ of ev^uattng a test item, fo either case hcme^r* 
the evahiati .ni of d test {oi J tost iU*m) involves making decisions concerning validity and reli^lity. 

It IS not our ptapu^it* htM*? to ttioroufittly review the literature on measuring reliabiHty and validity. For 
fiorm refenniciNl test^, |Hoi:Hi(Ltfi*i fur meajiurmg validity and reliability are well-known (see. for example^ t.ord 
md Novick. 19C8). Foi rritetfon f#»fen«nci?d tests, the measurement validity is retativety str^^tfcHward; 
however, at iNs time, fvw. <f any. pt ocedures have the status of ^nerallv accer*^ scrfuticms to probfeim 
surroundmci the measuremiMit of feljuhility. {See, for ex^m^jie, Pop^am and Husek,; Edmonston and 
Rdodalt, 1972: dffd Hambit.toti and Nuvick, 1972. ) Thus, rather than review the literature, we comider betow 
certain importafU i^ues concerttHig rMliahihty and validity, especially as these topics relate to the use of criterion- 
referenced tests in tnstf uctuMial -lytitenis. 



In most cjses^ there n no !ed!»o?itjWe external criterion for a critericm^el^nanced test. Usually, the only 
criterion umiet considerdfum an fntenYal criterion, namely, the iHstructtonal obi«:tive that serves as a tia^s for 
test construction. Canse<|uently, for a criterKjn-refcrenced test in an instructional system, the issue of validity is 
generally an issue of cofitefit validity. The extent to which a test has content validity is usually a matter of 
judgement on the part of tlie subject niattt^r specialist. However in some c^fs, the relationship between the 
obiecttve and the items tn a cntfricKi'tefettTnced test is so close that judgements conoemirig the cc^tent validity 
of the test are hardly necessary provided one accepts l^e validity of the ob^tives. For example, procedures 
dev€top€«l by Sormuth (1970) and Hively, Patterson, ^d (1968) produce crtteriiMi-referenced test items 
that are virtually isomorphic With the objective. 

This pretfmifirnt concerr^ vMh rcKite nt validity is the reason wrfiy Table 1 imiicates that achievement data do 
not provHie the principal knid of infoffnatuin for judging the v^idity of a criterion-referenced test. It diould be 
noted, however, that sucn data are sometimes used to calculate particular kincte of disi^imination indices for 
cffterii^vreferenceil tests (sf^e, for example. Cox and Vargas, 1^6). Such indii^ purport to provide an additional 
Icind of mformatKwi concerfunq cntenon^feferenced test validity, however, the interpretability of such indices is a 
topic of some concern {Popham and Husek, 1969; Brennan, 1972bh 



The determindtuMi of feliatufity tor a cntenon-referenced test in an instructional system is complicated by 
several factors. Fust, it 14 e^Mu raily u^'AHim/vii that classical procedures for estimating r^iability are seldcKn 
appropriate for cntenon f(?fefence/i tp^ts. Second, we usually need to estdE)tish bolh the st^itity of each test and 
thp equivalence of the several tt stii measur.nq the same objective. Third, many of these tests are used for two 
purfK>ses; to meastire stoch^nt {wih^rm^iiun* at wrw: po^nt m time and to measure chants in student performarK^ 
over time. This dual fur^t^on of nuoy cnTeoon referenced tests necessitates the determifiation of two different 
estimates of reliability. 

TTie atiseni:e of established priKr^'dures for estimating the leliability of criterion-re ferenced tests obviously 
complic*iti»s thf task of <»V'iiuat»r)q not of>lv the t* :,t fjut also instruction and students. In essence, the e valuator 
has three choices: {aj use lAdMCdl promJtJit.'s for estimating reliability, fb) use one or more of the newly developed 
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techniques, or (c| use some combinaticm of cf^sical procschjres and newly developed techniqi^s. ProbaWy. the 
atternative is the mmx priKtent, faut in my caw. the evaJuator must exercise consiiter aWe caution in interpret- 
results. 



f ideally, the reffabiliiy of t*.»sts should be determined before surfi tests are used in an instructionai sy^em. 
After all, it is difficult to justify cteqisions made cmcerning instruction and students unless one can argue that the 
instruments used f qr decisions taking are reMable. However, es|]^ially for aiteriw-referenced tests, from a 
pri«:ticai point of view it is of^n rmpossibte to estimate naliabiHty without using data collects in the ^vtronment 
of the instructional system. This is one principal reason why tt is desirable to fietd-tast an instructtimal system 
prior to its full tmplementation (stm tum!daine, 1965). 

CONCLUSION 

We sutecrifae to the view that ev^i^tion is a fundamentaJ part of curriculum development, not merely an 
aiipendiHie added on after th« fact. Thu^ decisions com^ming the c^iilecticNn and use of data should made 
before an instructional system is put into operation. We have treated here a model for considering such decisions 
as they relate to achievement data and time data. This mode! is ipnerali^able to n«ny nwctes of instruction 
charocten/eif as instructional syftems and involving clearly sta^ dtsiectives. Furthermore, many oJaptatfOTs of 
this model gre possit^e. For example, m order to avoid comfriication, we have tacitly assumed thr(Kj#ic»4t. *at 
cutting scores for ctecisions concerning inslructfonal effectiveness, instructional efficiency, mastery, etc., are the 
s^me tor all students. Yet. the ideas presented here, cpn be generafized to instructional systems having different 
cutting scores for different stucfents. 

Atmost invariably the task of evaUtating an instruciionat system is a very large data coHmion artd arrafysts 
activity ideaiiy consisting of repeated measurement of objectives at several points in time. This i^sk can be greatly 
fanhtated by the use of computers. In particular, it has beendjEjmpf^trated that time^ared computeis provicte a 
bjsis for efficient administrati<Ki of achievement tests and coitecticm of'data from sudi tests- Brennan {1972a) 
provides a review of ttie literature in this area- 
Most of the literature currently available concerning the eva^t nation of instruct ton«M systems treats the analy- 
>h of data pi imarily from the point of view of classical descriptive and inferential statistics. It is our Ijelief that, 
in the future, Bayesian staiistics and decision ^eoretic techniques such as those discuf?^ by Hambleton and 
Novfck (1972) and Cronbach ai»d Gleser 0^5) will ptay an increasin^y important role in ttte evaluation of 
instructional systems. These statistical prcKtecHJres are particularly appealing because they provide the potential 
for dtrectly incorporating many kinds of prior information and subjective judgement in the decision-making 
pr oce«< 
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h seems to that PLATO dnd TICCIT havu to be accouotaWe to some audiewigs whom I consider to b« 
very.mportant. The ftrimafY audience is perhaps students. Each of the ptesenters this morning used the word 
students sev«ral trmes. but neither one of them made an important part of their presentation, A critical part is the 
question of direct impact on students. Another issue is instructors i nd users. Vic confessed eartier in his presenta- 
tion that his stides had b,'m piepartJd for a dif fureni talk and that in recent months he has become much more 
sensitive to the question of faculty aa»ptance and the like, f wHitd lifce to sugaest that that audience deserves 
much more attention than it goi this morning. A third audience is administrators, the guys that have to mate the 
decisions about whether or not these systems can really five in a school environment. We have a had habit in 
research, and t am not accusing either one of these projects of setting up rather sterile environments for develop- 
mental programs, of collecting untter rather constrained conditions data that the psychologists and evatuators tell 
us we should collect; and we run it through a regression analysis and make a decision, and then tf ,e program sits 
So It seems to ir.o. another group to whom the projects have to demonstrate a little mere accountability is the 
ddf..m.strator, A fourth group that is no less importjAit is the professional community that « reflected by many of 
the people in the room here. » y " 

I guess the question that f would like to put to ead, of the projects with respect to the four audiences that I 
have ,ust mentioned is. "Are PLATO and TICCIT In their dewelcniment to be ,esp<»,sii« as to what professionals 
r«rK.Her to be th« implicit ,.e«fa of each of those four audiences?" A iot of issues were suggested this morf>ing. 
There is. as I have said earlier, a kind of mainline structure approach of TICCIT versus the more adjunctive 
suppleiiHjntal approach proposed by the PLATO people. Uncterlying each of those approaches are lots of very 
important questions about certified.-controlled authoring techniques versus a less coi.trolted target of opportunity 
approach that Jim was suggesting. I think that have got to to<A very carefully at both and collect some data to 
help us make decisions about them, t am not sure we teamed enough this morning to begin to even formulate some 
judgments. It /.^uld be unfair to require each of the presenters to explain to this group all of flie components of 
his protiram, but. nevertheless, 1 want to caution ^eopie against drawing too many conclusions on the basis of 
what we saw this morning. We saw some examples of course material. In at teast one case, an elementary example 
that Jim showed us. i think it is safe to say that the material was entertaining. { woukf say that neither in Vic's 
math presentation nor in the physics materia! that Jim showed us, did we see enoo^ to allow us to formulate a 
judgment about vrfiettier that material was pedogogically sound. We did not see enough to let us judge wrtiether or 
not that matenal has the potential to live in an educational environment. I guess what I am getting to is that I think 
these projects are certainty very complex. 

I think that «4iat we saw thii ..KJrning was just a couple of facets of what is really an enormous and in many 
ways, very groteso- ■ceberg. The tip o* c iceberg is what these projects are all about, and that is they have l^. 
funded to deliver, namely, high quality instruction to community college sectors in the United States-in only some 
areas of the Un.i«d States. In a couple of key areas, in the case of the TICCIT project they are to deliver English 
and math m^ruction. In the case of PLATO the areas are community college accounting, biology, chemistry 
English, and mjth. and also in the case of PLATO, an elementary program in reading and arithmetic. I would 
suggest that this is a very important char^. 

What I'd like to suggest is that each of the projects, in considering its responsibilities to the professionals and 
to themselves, begin to share information and ideas with this community <the present audience) and with other 
interested people. Especially important is the sharing of more details about the plans for implementing and mter- 
gratmg these programs into schools. It seems to m«. and I think that this momina both Duncan and Joe raised 
some questions about this, as d.d other pen»}p h^re, there are q-.:~tions about thei. schedules, and the like We 
know from preliminary contacu mat, in both projects, faculty acceptances cleariy going to be a piece of the 
^tion. There are problems of courseware development and coordination and intergration that are nontrivial Vic 
mcntio^i'd that the B. Y.U. attempts to identify a structure for English programs had met with some problems, 
ami I don't think that's particularly surprising What 1 think would be particularly helpful would be if B:Y.U. and 
similarly if Illinois would share wiili us some of the specific procedures that they have implemented to overcome 
some of these pioljlems. There is an enormous need for careful orientation and training programs for some of the 
ttetaded procedures on site management, and on the physical intergration of these systems at the pariicipating 
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sites. I think vw wotild aii do w^i to ccxisider some of th^ straie^es for explicating m%d mximmw^img 
capabilities that exist in t^ie particir^ttng schools, {nstmctors mii students cm fie a very rtcfi r^CHir ce tn program 
development. They can also be a very hc^lite and violent resmiri^ if not treated pr<v>er{y, I tttink botft proiects 
have to fofmulaie plans fur themiwjlwes, and, as l,^id, share with use some of ttteir i<teas for doing tf?at. I knew 
frcKit two years of experience with the Evaluation of ""Sesanru: Street*' and a year with the ''Electric Company" 
that the Uinor in schools today ts not to stand back and politely iN>plaud when harcM^are is br(Hi#il into a class- 
rDcNm or a learning center. Instructors are demanding drat more aitenticm be ^ven to their roles in these 
developments. Hiis is scnnething that both protects reoi^ize. t think that the faculty can reascmaWiy demand 
help m impJem^ting these programs; so I am suggesting that we ne^ new procedures to assist instructors in 
inter^tfoff and coordinatirtg these technologies into dieir d^rooms. i think we rieed more infmmation on the 
kind of feedback that is going to be pn^vided to fnstrwtors, Certatnly, m the case of tfie PLATO program in whidi 
sutHilenientat, adjunctive kincfe of uses are profwsed^ die burden is cm tfie dicHiiders of die devetoijers to provic^ the 
"hooks" and the "links** im the instructors and m^ateriafs. We haven't les^nedatxMit that yeL t think seiieriil f^N^opte, 
iiKluding Bob, this morning mentiwed the need for a formal pubik: plan to performed formative ev^uatkm. The 
c^ie^cKi of fiuality control came np: I dt^'t agree with Jim that it is an imie whh:li a tatoratc^ can duck. I think 
that the implementation of Pt ATO at the l*tiversity of Illinois has been hi^jhly idiosynoatte and in many ways a 
success. I dcMi't think that model is going to hr^d in die community colleges or the efementary sditmls* In fKt, as 
Jm has indfcated to you, they have implement^ a nmre fc^mai prooK^ure f<^ eteimntary c<Hirse develc^iment in 
their own environment. In any event, i dcm*t think it relieves die pr<4ect of tfie resfK^nsltNf tty to pi ovic^ plans 
and milestones for the review and validatimi of ccHJrse miuteriat and infcMrmation abcmt how the cyiatity of lessons 
Witt tie controlled. I think some empiricai evitfence is recHiired; it on^tt to be ctemanded to suppm some of the 
directions that are bemg tmpt^nented in eadi of die vefV differt^t strate^s we have t^ard about this morning. 
The educational community can justihably cfemond evidence of the effic^y m die apprc^riateness of some of the 
courseware development api^oaches that were cfe^ited this morning. Finely, t think that it ts time lu piessure 
each project to release operational schedules for the transiticm of all aspects of this f^ogram to tf^ sdiools. It is now 
almost October 1 , and these promts are supposed to be deliverying instructitms oy Scptemt^r of 1974, and lots of 
things need to be done. To insure that the projects dcm't ^itte b^fcwards into the schocris and to insure that neither 
me runs the risk of being judged a failure for the wrong reasons, I think it is important to move <xi to some of tfie 
stages that 1 hi^ve iust be^n to suggest, f ^ess vtrhat Td like to say just by way of ctmcli^ion is that t think that 
tK>th projects have gone a long way to date. I think tfiere is a long way iO go. f {t^t want to remind everyone, 
siiKe it wasn't a big part of the pr^ntations morning, diat TlCCf T amt PLATO hai^ a very* $tn»ig obligation, 
I believe, to a community of users; and the two primary fi^oupSwin that ccmuniHilry are students md instructors. I 
think that we would alt do well to pursue some of the things that t have been ^i^gesting. 

OL Ernie, could you say a couplt- u! things at>out your own plans for the evak^tic^ study? In other words, how 
it ^apes up witn what is projected ahead? What are your plans, arid wt;a4-st€t% lie rf^ad? 

A. Yes, I can say some words. The evaluation plan that I rerentiy prc^K>^ to tfre National Science Foundation 
did not in any s^if icant way get uX, or propo^ to get at, any of die Issws diat f brcHj^t up ihh nKmiing. We 
dfdn't know what we were ^tting into origtnaliy. We dicki't know w^at PLATO and TfCCiT wem like. Doing this 
evaluation is a lot like ^KJking a marshmallow. You know, ycM poke it and it H vimm and it ^ms to conform to 
the poke and you take your finger away and it just pops ba::k to where it w^ t^fcre. If ymi poke too hard, you do 
irreparable damage. Then people say you are the evaluator. You are not su{H>os^ to structure tf^ project. You are 
imposing yourself too much. Yuu are fcieing too heavy-handed. In any event, we are witling to strike a balance, and 
what we learn tbroiir^ our regular contacts with tfle participating sdioc^ h Uidt diere are whole classes of social^ 
institutional variables that are really Critical to whedier or not TlCCIT or PLATO are every going to become market- 
able commotlities in the ^-^^'^ctionai field. To t>e sure, there are issues of ^levement, master certification and 
placement that are important. This requires some kind of formal data collection instrumentatiun, a lot tike the 
thin^ that ETS has used in the past; also lots of new instrumentation— topksal tests and criterian-reference tests 
mentioned earlier, an^l the like. But, we are finding, to use the analogy I i»ed before, that is just the tip of the ice- 
berg. What really seems to count in the schools aw is^es that relate to the role of die instructor, the way in 
which this specific technoloqv k <joing to augment, repl«:e, supplement, or whatever, his role. The evaluation desi^i 
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» becoming, we think, imich niare t^sfKH^««, in ttie itnse tiat we hai^ implMrmiteif ^ehnlq^ fm re^iiidHy Inter- 
viewing partiGipmis and hav« flat a sef fes of ^asikmnain^ on instructor ^tivilies, that kind of thins. But 
what need w to stay m refpuktr cont^t with msi^i^rtm^s vnho m cm identify m particip^rts in dw progrimi *nd 
^th ite(ijtmti»nt chairmen, fmitect us aqanst oreatmci an artHictal Hwcr i ^ of 'liave" and "he«e*m9t$^'^ ^ 
gpnerahzc this aijKXt of ^ mr^ikm so thiat vwf m » fot$ of pe<^ in HartUripattng ^oi^ havent be^^r^ 
tmi ked or identified d$ f^rtH:lp»it^ So th« a^$^dr« dev«fof:fifwnt in both propcts stiH h» a long «»ay to gol We 
haVen*t i^e vwy far m terms of da^t<wiing any criterii^-i^fGfimG^ or tomc^ tes». 

Q. Are you looking at the (^ocess itw^f ? The point eante up mtier regardif^ *e number of hours required lo 
itevelop one student hcHir of material. Jim mentiwied 26 hours^ \ betie^^, as a reouiremmt for ifevelopment. That is 
ot .viimiy an ater^fe, at best. If Ksm looks at tto spectrum of authors and tyi^ of materials nee^, the diffeieiKes 
h% ^^fvetopment time am some of tfie figures need. Are there some cm^tte deta lAKmt the proKS^ iMStd and (rians 
to get data so that they are being affected now? 

A* y»»s, we arv very int^^ted m proce^ ^lUe i% comment, I am not wttHns to te« v^t I tfdrdt the 
authormg requirements on PLATO ai^. For anyone tftat is interested, if t can use your phcw^ fine, I havea pr^^ram 
wnttun wdi the h*4p of scmw ETS md ^mehc^ from tiie Li^mm(^ at Wifum cattCKi dat^^ Mfeate 
Jdo* <n<i daily at dw activities of 44 mtthoit on ihn» system. We are geftir^ a good hmd^ on tf^ amount of tinw 
they spend at the terminal doing*different kinds of activities tike ^itinQ and running in student nmte.wti the Hke. 
Wi^ have ai tot of cbta being collected <m d contimring fc^s. Utat is why f am not mUU^ to give yon an ^krate. i 
thmk the estima^ Jim gave you is certainty true for certain kkicte of authoring activitm I woidd su^g^t on tlw 
Imm of my activities that 26 hcKirs of i^thortng pet hmir of terminal time is somethtt^ that you cannot rei&c^atrfy 
expect from many Tutor prc^^mcrs. The pec^ile wtuj ha^ been aroumi die lab for many years and ha^ beccmw 
Tutor expert can, I am sure, get half an hwr of mat^ial or forty-five minutes of mjaterial in tvventy^ix hmus. I 
think what vne are finding, and this is m f ^ doain^nted in a IH. Alt> evaludti<^ lefmrt ^vtiich is availaUe fr<mi 
Al Aimer in the Illinois lafoc^atory , is that it takes ^AxHst three mcKittis of hdf-tiiT^ prc^amming for a new Tutor 
progtammer to write |iis f im lesscMi. It tak^ three mcmths of h^f-tinie pro^amming for an autttc^ to write tiis first 
lesson -probably a similar amount of time before he refines atHl au^nents and ctebugs, aiKl «rfratever« in a te^on to 
tlie point wlie*« he is ready to say this, "It is scwnething 1 am wtlitng to (Kit up to be a dies of pec^/' Tliis 

IS certainly more extreme than Jim's 26 hours. 
* ' 

Regarding funding, I cuess PLATO is^ptting a total of SIO^OM^CHX}. f rom*NSF. A development like tbm. which 
n^roves the industry iritead of ttie tlwory that is av^lat^, w<Hitd cost hundr^ of mfilsons of dollars if m^r done at 
a university. If you look at the fibres the deveicHHn&it <u>st of the LP record or a>iored TV, diey wmild 
muc^ greater, Whatevw we ctevelc^ in technology is expensive. 

Deveic^>ment promts place a tremendous pre^i^ on the pec^ invc^v^. I know the TICCIT c^oup is working 
day and night, and they and the PLATO groiH3 are the imt in the tnistn^. There s^ms to be no way to hurry these 
developnwirs beyond a certain rate. I g^ss t would irgue with BcA C^ws that in ctev^c^ng ^unw s^tems, dw 
increases in i^^essure, after a certain point, does not make performance go up The pre^re that exists comes from 
Interaction with professional j[^fs, &) far the pr^ure, in the case of.PLATO, tms f»^odiK?ed what scxne ^I 
is 4 itfChnologk:al si^xre^, and p^hc^ it is. Now, 1 think it is time to demOTstrate diat the system c?n al^ become 
an fKiucational and instructional success. There are sweral milestones ahead which these two projects have to readi 
jf they are to develop systems that ^rk in the schools. 



156 



COMMENTS ON THE PRESENTATION BY DUNCAN HANSEN 



Robert J« $ei€Nt 
Human R^uic^ Rc^aieh Organization 
Instructional Technology Group 



t. As I notetl relevant to any computer utagfi ifi Jnstruction« the use of task mirtys^, statement of trfi^tives 
in ^trnvemt^t t^ws. crit<Kicm wfetmusmi testing, etc, all haw ted to In^N^oirennenis 4n course of instrwcJtfoa In moi t 
imtattm «t ts umtear that the ctwnpui^r i^ayed a si^hcint role. Ami tmfeed in tk. Hwsen^ ptesentatJon he 
citet this point. 

2. This, in turn, highU^ts tt^ profatem of t*ie ^Wi^mdtfy i^K^k fin^Kim ^iei»ft»e n^re the 
leptication ^ extemicm of results fptten in om ewironfmnt in ano^^ instrwtiimirf envitonfmnt Tlwaxwifrte I 
Msed at the cunferem» wsi ttfiat of Hurst's su<xessfui miteiiHim wifli aucfto^iKj^ ai cm^i^d to the f^lures indi- 
cafisd by Dr. ^aun and others. The p<^nt here ts that untess and until explrat instructitm^ strate^ ana detsmitned^ 
the definition of innovative technlcpies by potential imn are oftOT nus^en^ lestfic^ to the sup^idri ihatariab 
or nwdia as opposed to the pedagogy behind th<»e technoli^es. 

3. Extreme care nuisf be taken theref we in docomentir^ the c<»iditions m^d ^ siratepes for administer- 
ing the innovations in each environment. 

4. i foumi myself in gener srt ^eement witfi l>. Hsmen's i^s^on that CISi if equhrcrfrnt to CAI tutorial 
in effectiveness shmjid be used becwse of iti lower costs to develop and to in^ement as a syst^ for {r«5tniction. 
Hcmever, my caveat is that the specific instamre of obt^ing sudi an etfuiviri^ice may well be the r»ult of a ladt of 
development uf a clever adaptive modef of the tutor in the CAI format 

5. In general, I a^ with flre stated n^ for ade«piate fite stn^iuies awl inquiry OM^iuting systenw and 
take no issue with the pre^^entation of the softwwv pc^^cm of die p^r by Dr. H^isot, 

The folfowing points Illustrate with cotKrete e)wnples si^ne of *e Iknttations of tk. ftonsmt's t^A^ring 
gener^ assertions, f refer ^aecificaHy to tfirro Florida State Univmity dooin^ts dted « ^qqiK^ve by tk. ffanserk 
f^wwer, I find them ambiguous at b^t mf6 in some instafu:^ cwtraittctory with his a)ndusi<m. Therefore, I 
(isa^ee with the broad generalimiDns from ttw r^arch d<»ie at the FSU latK^ratory- Some of thew pdnts I 
irKiicated dicing my presentation at tfw cwference. A^in, ^ f noted, it rs fn^ra^Wfi to ifed with the vast num- 
bers of studies conducted at FSU and referral to briefly in Of, H^tsm's piH^er. htowever, I will simply ccmm^t 
on a few points in detail to illustrate the over-generalizations whtdi I feel are part of #ie Hansen presentation. 
Wost of the reseanrh that led to the very broad general ixatiortt pre^ted in ftensM's p^r Evolved ^lort 
instriNTtional sequences on the «Kder of anywhere betv^^ 45 minutes to m hour wd a half wortfl of iTOmKrtion. 
It IS not at ail dear that the material dealt with the hierardiically organized and ridi materia of the usu^ 
instruction^ environment. g 

t. In the discussion of mxmty in learning, Hansen states the fi«lowing: "The important results imftcated 
a consistent inverse relationship b<?tween state anxiety and learning, differential effete of h-ait »txiety . . Con- 
trary to Hansen's assertion/the TR which he cites as suj^ort reveals that state anxiety and learning are p<»itively 
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njJated in sotw instances and negaltwtv mia^ in qttim. (See esfiedalty t^gi*s 2d and 30 of th« Lee Rlwm nch. 
Repmf^ Iff acfcHtiofi. in fh^t study in att cases wrfiene n^ait MiMiety was significantly rotated to the crtterfon 

scwes ill Jeartting, the retalionslHiJ was fiositivf?. In other wortfc,^ iituAngi are nm nearly ^ omsts^nt as ^ 
«?ncralf nation made by f^. ttensfen would s€em to jndfcaie, Anottitsr study i^{a^ to the telatioRShip b^tw^isn 
curiotfsity a»id anxiety also was ci^ in Hwswt^ t^esentation |FSU T^. Altema. Ho. 341. 'Hie tetatimi^p is not 
at all as eon^inctmi or clear<:ut as the Hansen ptesentation indicate- For ex^npte, on pag@ 2S of the a^iienoled 
Fee*. Atemo., corref ations between a state curioc^Jty scale, a semation seeking scate. and an pnxiety state scale 
are tKesent0d with no sf^fu^t ccnreli^c^ai ffmting be^rveen ihe %tat$ curiousfty so^e (SCS) ^^er tfw 
sensation seeking «:ate (SBSSJ m the A-sme scale (A-SK Hc^wi^er, the intefpretaticm ts^ven as^ foltowsi **As 
expected the SCS^^ not found to correlate signifrc^tfy with the SSS. The expected inverse retetionsWp tietween 
Aetata and the SCS was found, although tWs corielation did nn? afiproadi signihoaiwe/* The sad tWng about thh 
ttiisinierpretation is that tn aikUtion to tadkingstatistioal me^Jns» tfte numeric^ y^ue of tf» ^ ami SBS 
com?lati0n is +^ J 7 when^ the nun^rH:ai ^ue of die ^ and A-state scate ts sm^fer, ---12. 

V 

2. In some instance, such as in FSU teeth R^mft Hp, 12, where no differences occurred between teacner 
control versus system control seifuendng of material. It Is mpt at ^ dear that the conditions of measuremei^ were 
such as to fewd imfionant Student-materials interaetrons which might have led to different condwions frwi the 
no significant f^fference findings that o^ned. In particular, t see oi> fiath ar^lys^ m tfie <tata. Yot these are 
essential to determine if students have a self-sdectkm treatment tiiofc Sie f^escrited ^;l» in ^y differ^t orcter 
from the tasks which wen? ^x^^^iKred fw a second grw^. In iK*rfition, itm namn? of the interai;tlw$ betm ^fw 
ttent and materials as indicated above is ^so ^nbig^oM v^th resf^t to tfte experimmtal t^atment omditic. 

3. In Hansen's presentatton, the Tec^ R^wt by Lee Riyers is cited as a mai« sftKiy of affe|>€f^ model- 
ing at FSU. The resists i^e generalized <^ 20 of the Hamwi presentation m fcrtl<^: '*The mdm of mitoimes 
in temis of superiority were ac^tive model strategy, remediatic^ for all strategy, a learner choice strategy, and a 
no-remediatfon strategy/' Yet if one I<m4s at the TR in detail, we find ttiat ^e fesults am not at all as general or 
dear as are implied by this staton^t For examf^, it shotdd be noted that dieie wem two levels of materials 
presmted for instruction. The first four concepts were ^rt 1 of the mstruetion. ^ diis part w» dte simpler 
material ^>d nwe familiar to the students. P^rt 2 cc^isider^ to be mtwe iedmic^ material mme difficult 
for the student and less familiar. There vwre 9 eoncepts^teu^t in dlt« fow m the first part and fit® in the second. 
Only in Part 1 did the adaptive nKKtel ^foi^ show signifkmitly ht^er performance. However, as the author notes, 
these results did not hcrfd in ex^y that way for the seccxn) part of ^e instn«:tjon. Within Part 2, ttie fegression 
model s^oup and the alt-remediation group performed sispi«*»c»itly better, . but there w« no significant cBffcr- 
ence within the two sets of ff-oups/' Ip, 42, Riwrs, 1972} So In the mote cfifficult part of the task, tfie rfl-renwdia- 
tion ^oup was as ef^tive as the Motive mcKiel p-cuip. Furttemme, the mem tinw fat the ril-fenrnffation ^tiup 
exccected the adaptive model g^mip by <^ly 5 minutes. Therefcne, the question of tf^ f^^tii^l Sfyjrficance to the 
outcon>e. vis-a-vis using an^lapttve model stra^gy^ is question^e. For exampte, hijm mwh time did it take to 
develop an ^iaptive model as c^apmed to simf:riy using ^1 tfte remediation^ caf»tNlity availaUe in the system? 
Moreover, the rest of the resists indicate that the learning ta^ it^lf ^ov^d a deterit^aticxi in );»rformdrH^ in 
Part 2 of the study to the point i^ere it is of qt^tionabie utility to talk atKHit tfie t^k ^ a rermsentative 
instructional task tiy the time concept 9 W£^ reached. 

If I were to sum up the results of tf^ study in terms of the orcter of outcome, contrary to Hansen's inter- 
pretation, I w<Kjld note that concept by^om^pt analyses r^ealed that in four <xit of the 9 cements there were no 
differences an^gst the tieatment conditions and in oths of the other ramainrng 5, ttw ^Irert^iatiDn grcHip w^ 
superior and the superiority of the adaptive model was principally r^tricted to the more foliar, introductory 
material of the instruction. Finally, I note again that the author ttecribes the t^k % lasting r<Hi#ily an hcxir and a 
half rn leng^- 
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MINI COMPUTERS IN EDUCATION 

t 

Ludwig Braun 
f^of^sor of EteclrkAt Sciences 
SUNY at Stony Brook 



tli^ presefiiafKi^ at this meeting itesorihing tiie Ctaitmouth Gcmif^er System ami^e PLATO and KCCIT 
sys^fti^ are impressive ttemomtraiions of the iKmer whitS% eximiHiiers tan britig to beBr mt educatfonai 
process. I askeii for an o^jpor tuniiy to oHnm^t tH^tefly cm ttte txAe of the mmi<xmi|Hifer in eduoati on be€ai^ 
I believe that the u^oeeet^n^ of this maeting would tis incomfrfete w^iout recognition of tN$ riMe, 

Ctearly, thera are many computing jote fm vi^htcH the minl-ami|Hiter i$ mit adequate. The anomvius $ikU^- 
soence data files which the OartnKHith student h^ available can not be iniirtemented <Ki a mini-omiputer* Cartain 
kinds of very comptex computations and certain very desirabte languages can riot be Imptenwiied iHi a mini' 
computer. None the te^, there is a great deal which cm be dc^ In the ctesroom with mini-ccmjpufiars. 

There are two re^ons why I feel that the mini "Computer is becoming an important compof^nt in the educationai 
arsenal. First. mini-ccMnpoter manufacturers and soft -ware Nhji^ haw ctevelop^ mx6 are oonthiymg to dev^c^ 
very efficient and very user-orienteti software, so that the mini is mu<*i ^ier to use than vm dw case as n«»nHy 
as five years Second, the cost of mint^mputers has decreased cfaramatierily over the 1^ decade. The price of 
thet>i^tal Equipment for example, had decre^^ by a factor bf 2 every three yems« on the ai^ags, during 

that period. Once large-scirie-inter^aticm hit the mini^mputer, the prices shtnifd again drop drwiati^ly. 

In the Huntington Computer f»rcHect, we haw had e)q>erien€» fm at»Hit six years with mhil*cmnfHiters In 
hr^ sdrools. In 1967. when we were formulating the purpc^es of the Himtin^on ComiRit^ Prefect, ^thur 
Meimed suggested rather strc»tgiy that we should explore the of the froni-computer In tf» dm^oom, as well 
161 Ihe use of tin»-si«red term^s into a larger ix>mputer. One of our tm^m a^sfMishmmits, In feet, tias been 
the demonstration of tte value of the mini-comfniter in the^^high schocrf. TTie two h^ ^oah which started with 
a POF-8 six years ago %ti\\ hm^e them and svi^ar by them.^^Thc^ irsist that tf^y would fK)t have tlnw-rft^ng, 
ixKrause of the communication problems whi<^ can ansa. In both oa^^ sdditt<mat memory anf additicmal imi- 
i:4>erats have tieen added to the system so that the systerm now are much more jKiwerful thm dwy were in 1968. 
This imteed is we of the attr^rtions of the mini-computer. For tess th^ $ZO,WO a i^^wA can have a complete 
ei^t-terminal time-sharing mini-computer. Certainly the da^ of prc^enn whidt can be harKfied In wch a config- 
uration is not very Urge; however si^ificant learning activiti^ be run virith sudi a machine in naithamatto closes, 
for example. Furdier, for just a f^ thousarrd dollars a year, additlorwl components may be acMed to bicre^ the 
capability which will satis^ almost any high sdioot te^ef w stuctent. 

t should like to relate also my experience with the cm of a mini-computer in undangp'aduate oourm in system 
dynamics. We have a POP-8 with only an 8K core and a simulation li^i^age callwJ ISL (devek^ied by Inter-cRrtive 
Mini-systems, Inc. in Kennewick, Washmgton). With this simple systmn (which cim be raplka^ for urnter $10,(X)0 
including software coste) we can simulate dynamic systems of up to KWi orcfer and OKitalning Vme^arylng and 
rnwifinear components. On bench mark tests comparing (Hir POP 8 plus ISL wt* d ming CSMP, w€ have 

obtained conrrf3arable accur^v and have obtained solutions with comi^raNe anounts (3*U time. At Sti^y 
Brook, iw currently an? using this fnacHine m an under^aduate course in bio-en^neering. The stoctents are study- 
irtg the dynamics of the respiratory system, and of the tr^smission of heat from the eort to Ht^ skin unc^ various 
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environinentat circumstimces. I can visually the time (in the not-too-c^tant hiture) when the teadter of Physio- 
logy or Biology or Engineetinft e.g., will wheel a mini into his das&room, and, using a langua^ ilke ISL, wiH 
explore with his stutients the dynamics of systems which are far nore complex, and far more realistic, than is 
the cme in most classrooms today. 

OiHj Other iM^t of mim^computms which mifKjrteftt m etk^tkm h the advent of the n^i-time^ring 
computer, Hewlett-RwAanI mth its 2000 and 3000 series macWnes and O^taf Eotiipnwit Corp<»atlon with its 
TSS a and PCtf>-1 1/4B systetm atr^Kfy haw tad a dramatic Uxtpxt m sc^Q<]rf4ei^ eomiHitif^ FVobaMy the 
bmt exampfe of this exists at the TIES f»roject m Mimi^pcdis^ althwgh similar activititfs txht in Colorado, 
Northern New ^rsey. and Mass^usetts, to nan«r just a few* 

In conchision, I shmild like to say that mini^mpy^ hnve a potemiciny very important role to play in 
education because: 

I . Their iow price bring aTm^nittng power into e«n ttte poor^t schoi^ system. 

Z The cwnputing pm^er of the mint computer is imrreasing dramaticaiiy tec^use of toth harcbvate ^ 
software improvements. 

3. Aln^dy the capability is entirely ai&auale for many edi«»tional fUirposes at th^ &oliegB level, as well 
as at the pre-coite§@ level. 

4- Their size and flexibility (comp^^^te to dte flexibility of a portat:^ 16 mm sound projects) makes 
them very attracti^^ to the edt^tw. 

5. Their lack of dependence on communicatfon systems (omviar^ to the d^mictence of time-shaied 
systems on comhHinication} is very attractive partly bet^^ convnttnication systems are re^KK^iNe 
f(K a sometimes un^^ptatrte deteric^atic^ of system reliatrility, af|d also becat^ communtcatttm 
costs frequently are comparable to romputing costs, arnl can e^ exceed the cost of Hie computiiH^ 
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ENVIRONMENTS 
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CHKJAGO^S TV COLLEGE: A TELEVISION EtASED OPEN LEAflNING MODEL 



James J, Zl^efi 
Dean of T.V. College 



Ins u cimintuti UuH of cihitatm"^ well uf critfcs df the arts-to eviihiate a prt^^ without fi^nl tc ; * 
tiiijt^t:tivir^ fht' KHOf<H:t'5 fifanotjrs an<l maiiagtJts set for themselws, aiid without regard to tfte Mmitstiotis under which 
rhf? nnuui$tti^ nuia wwk. ^tow^ldays, for ifi the worki of tfi^fructional tetevfsion^t H only to be expected 

ttut fhf intir will cofitjt?nin an imiructtonal TV ^fo^ect, or express misgivings abmit it. if it does not employ the 
t?K4itMHivi.* pfmfiiction todini(|4H?s of the te^iskin o:>ittmertiat« ewn thcmgh the f^oject has go^s qaiu diff^r^ 
iwm ttu^if of ihe adverti^r, <s mmiiyg at a na^i^ow largRi autlience with ^aeidfii^ f^s^mi opiates vrith a 
huik|t?t fur fH»lcHv tfw? Six miUiuit (tollars of S&mfwStr0^* 

True, ihtnf js, waitinq to Im %vtmi a vast ^e-sdioof aifdience, weaned on fmielic TV advertisinQ, p<»sessin9 
lt«> Umioii imunuon smn of tN^ chtid, and often suffering from the ctiftura) deprivation that comes from 
inH»wtf<shtfif homi! ami family oxfwrience, m* atttrntion of this audience cannot captured and f^ without a 
mossiv*? mufif siMisory stMnuKiTion. But e^h instructional bro^teast effort nnist be jiKJged wi its own terms: the 
atidiena? «t solfH:ts a* its target; the nature of the educatiwial s^vtoe it fmivides this audience; the fectors that 
ofHTdto to mak<? it "ciist effective''; the instructional "void" t^^t would rwult if ft yme mit oper^e. 

TV CoHotfo tha operation ctescntHfd in this paper ^ts a long established project. It is b^t af^oiN^hed, as it 
has hwn HI sijvifral f>ulilish*><i sttwlies during th^ eighteen years of its history, as Ofl»--l)y no means theon/K-model 
of an ofien broadcast Iwed wfucational extension. As a model, it hm been consistently effectives effectiveness 
is measured by thf} mstruments familiar to educators in a lar^ metropolitm area, ixwitaining some six and a half 
mtllirm fKipulHtion. Every school year there are enotK|h viewb.s who take advantage of TV CoHe^ instructional 
s<jfvicf!s in makf the costs of ofwn television production and b4oadcast mar^^irtile and "effective/* 

Before f doscrilx- the ciemetHs that have enabled Chicago's TV College to survive, while similar projects in 
other iwrts of the nation have provetl to l>e shcxt lived, an import^t disclaimer must be made a part of the recoixi. 
TV CoUeqe ailministrators and production personnel cte not claim to have exploited the resource of the television 
n^nhum if self to reshape; instruction or chan^ learning styles, althoi^h over the past fa^v y^rs they hav^ made 
serious attempts to improve overall producticKT tfuality and ^d the entertainmwit values to instructimi n^ed to 
recruit dmi hold new kirnls of audience. 

Now let us look at Ihe elerwnfs that have given TV College its distinctive shape as an instructional service, 
dt!icu^s tin? ov<*rall miina<^men! of instruction, and df?scrtbe the measurement tedinicrues which Supply information 
diOul dmi help cofitral student performance and costs of <H)eration. In ^^rticular, we shall looking at 1) the 
oh|(u:tives of tlu* project a$ tht-y have evolvetl over an ei?^teen year period; 2) the instructional system and its 
(ivef.fll iiesujn: 3^ thv TV Coltetje tart^t audiences, their neetls and their performance; 4) the cost factor. 

TV 0>Het^ IS an extensiut^ uf the City Collect's of Chicago, a two-year uublic community college. It is now 
fwrt of a unit tent^itively CdHed the Learning Resources Laboratory, a recently estat>lfslt^f uml that, when fully 
<levt'loptHi, wiW provide a staff of learniny specialists anil varlc^/ of instructional materials for the seven colleges 
makinq up the City CtjIteQes of Chicago. In short, the purpose of the Learnir^ Resources Laboratory is to futther 

t!HH!v,4ti<»f.> HI itMdiHt!| <hk| it.'dfntHq iit thr commiifiity colle^|i.' hy^;\, 
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(H^en cotor teteca&ts are prest»f1ited it^ cityS fwMic tdevtsiiKi outlets: VHF Chmmel 1 1 umi UHF 
€:hanti&( :K>* Weekday daytime and Sunday morntng bmadl^astft atie rie^ fay Chamtel 1 f : evmtr^ faraoiicasts are 
c^rHcd by Channel For the H>73 74 sdiocrf year, however, the idtedute on Otanmrt ^ \fyilt be Sif^fiemied, sinee 
lUt station's ^i^ul fJimiutth af»/f aiiye an* unsatisfactory. It is hu|ied that wltfiin the next ei^teen months tfte 
thaftnii^l 20 antenna wiH tm Ii4uved to a rtett^r tocatiun and its trmtsnittter replaced* 

* ■ / 

TV Colter tetec^ujp^es are also available to Chicano area r^fdenis In videoeas^e ©CWY IMMATtC, 

recordings in certain Clwsa^ Pdiitic Librar lies, as f»rt of a rec^tfy inaus^ated f>ro^ OBited^fi^ Uidim^^ 

whf^ is fimded by a ^ant frimi the Mlimns Board of Higher Educati^. Students can yfew tet»:aiSlsi it n^lsrty 

scheduled times m yroups^ m can arrant to iriew casi^tes cm an individual basis. The tatter option will liable a 

student to Ci>mi)tete a <^urs#f m It^s than tht^ ctinyentionally prescribed time< 

it is also possible that wi thif i the next two years TV CoHege n^teria^s will afas^ be dkrtributetf to ^ 
oolltfges nf thi*Cay College system, as wdl as to health care institutions^ j^i^mment offices^ and indti^riaf sites, 
^ «a the f mir ^annels of an ITFS system. The itniversity of tUlnois - ChN^ip Circle Cann^ Ns ir«licated its 
mention to^^ake use of the system for iirof^ionai in-service training if and when it is in c^H^aticm. The license 
wtO be apufted for by the Cdy Collegt*s o^ Chkraao. 

/yministrattve direction of tht? Learning Resources Laboratory and TV College operatims and planning is 
entrusi^d to an Executive Ctean. He wN;irks under the direction of the Dty CoUegra' Vice CtiarK^for for Fa<»ilty and 
instrg^rtfon. 

/ It is fitting, of course, that TV Colie^ be an extension of a <x>mmtinify college, stnM the ccHnmimity college 
by its very i\atufQ is committe<l to instructional innovaticm mil servt^s to cttijeens of all agas amd bsK^p^ounds. 

^^Itmrtional Ob^ives 

/ A valuable UNESCO mpnogr^, The New Medki: M^tm III f cftica^orurf fHaemer^ (1967L organized 
/ around the five educatic^ needs that can best be s^ved by ITV. Aithoii#i tfie comf^ters of this (A^ltcation 
were concerned primely wtth attackir^ urgent educaticmal programs m the wtwid's developing nations, the 
neetls they identified are ones detnafiding careful attention in the developed mitii»is & well. They are m follows: 

1. the fieetl for iinprovmq instruction in the ctassrocwn: 

2. tfie need to teach lUine who are ^d wilt be teachers of the youi^ and old; 

3. the neeti to incre^iso and ^Jte^ literacy ant' the skills of living in an urban, 
technological society; 

4. the need to proviilc continuing education for adults; 

the need to provide extramural extensions of the school ami OTltege. 

Since its beqi nning, TV C<)ll«^ has been ^atiy concerned with all but one of these needs, fts yet, 
unfortunately, it has done iitttej to offer training in literacy and the skills Of urban living. Nor h^ each of the 
neeils tn^eti served well. As for the one need hardly served at all, I Shalt indicate how TV College hxspe$ to reach 
out to those whose needs are not for formal adult or higher education, but for b^ic Vocational skills. 

Adults who comi>lete all, or part, of the f init two years of college by watching ^levision at h(we are no 
longer news in Chicago. What started ah ^ptemtwr 1 956 as a twld new dei^rture in extending educaticxial 
r^portunity, a three year project urxlen/vntten in jart by the Furxl for the Advancwient of Educaticm of the 
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Fwfd FiiumtaticHi, has toiig &irtc<^ Wendml into the? ediiGialHmai (ami^cape of the CMcago metrapcirittari mm. tn 
th^ well its of m e,Hlt«r |H»l>lisilieil TV Or^tlege rei^rt, the trtaf years from 1956 to 1S»1 ''sh€»^ that a funim 
cuHtH^? f«^«>gtam wn off*^»»<l Hfrnivcly oiiaf>t?n«Ghcuit TV" - i^ithout aiiy sacrific<$ o* tmtructkmai q^tity. 
AccfAHiitiini a^incnrs, pruh^SKiital a^cmatiom, oiitetj^ ami UDi^rmities tjhr^i^^i^t th^ Midwest a^oom^ credit 
^tiiiHi thfotigti TV Cui(t;qt^ wrthtnil qu^ttoin 

It iiteo (|tik:ktv l^tme a}i|iaront that the Chtc^i^ area, sen^ erfucattor^t VHF and UHF chanrais, 
cuntaiW'S a vii tiwtl^v incHhaustlWe audience of mature, hig^ily motivajted peofite capabte of o^im^ileting cottage 
ciiurs«*s by study iii^i on tH<Mr mnt. Oice the Ford Founriatian yant had been exhrnisted in t960, the General 
Si«Mt?r»«uerteft*iit of Cfitca!|oS PubNc Sdujols, who was then the chief administrattve off of Chicago's jM«»or 
tojlei^ system, nioimmtHHU^d that thf^ imttvcttcNiat teteviskm fj»^o^am be continue $nd sufii^rted tn Hs entirety 
by caxjiayers' fumh. Stiice when the City Cotteges^f Chieagocame under the oontrot of its own Board of 
Tri^stet*^, th*f itTstriictfwial tefevision service has rec»i^ only the wami^t sufi^iort from itus central ac^inistration, 

Ovf?r the past twelyc or thirteen years, TV Cotie^ has r€^^{y poiUsd yiew^s' opinkim of its instroctloriaf 
s<?rviu?. An overwhelminy majority r^Kwt riiat d^y like ttieir TV coin^s and, even more iit^rtant, that they 
enjoy sttitlying on TV. The result of one questionnaire distributed to TV Cottets^ students who h^ gcma on to 
convent ft>n*tl stiHly ir» four-year coHe^ deserve mention. Most of the tt^ree hundrmi resiHimtents stated that they 
}^arne<i just as mudi t>V TV as they did in the conventiondi d^room, Hiey stated furttmr that the ^-ade^ they 
made m their coriventiunaJ courses in four^year coUe^ Were ^ut d>e same as their TV gr^es^ not app^ticrt^ly 
lower. Furt>ter-and this *s not surprtstng, simse Chic^^'s TV College cnciur^ are produced for <H3en'Csrcuit 
lit oailcast -they a!i judged their television courses to be tetter organii:^ and more effectively presented than the 
conventKmaily tauyht courses they had taken in the coltege to which they had transferred. 

The first three years proved that a metrc^mlitan ar^ the size of Qiica^, with a potuntiat viewing audietice of 
some six to six and a half millioti people, contains a virtually inexhaustiMe wdtenoe of mature, able, and hi^ly 
motivated suuknits ea^jer to enroll as creiiit students in coltege-level televisicm cx>urses* 

Pie record speaks for Uself : 

Over 1B0,(XK} individuals have enrolled in televised oour^, with most taking rH> more than orte 

course; • 

-Of this total, some 80.000 students were enrolled officially in the (x>lle^ fm* cre€iit. 
Another 70,CXK) students have enrolled unofficially as non-credit students; 

- An averagt* of tO.OCK) viewers watch every TV College program; 

- TV D>Me^ IS on the air an averse;^ of 25 hours weekly; 

-The sttttfent r L»tefttion percenta<p; {that is, numtjer of students who complete a semester's work) 
averages tietween 70-75%. 

Here art? d f*;w other hic^liqhts: 

To flate. rfppf oKimately 2,150 students <f aduated from the City Colle^ of Chicago took one 
s^westm of theif work by TV, 

- A siAitole proportion of TV CoM^ students also take conventional courses on campus, 
AtH'i«*«<njdttMy 40 .' > of TV st^iKknts plan to U^ach, 
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ihe tmtrwsttomt SysMn 

Ourkuhim Until m*< tnufy, what was produced and tdecast was targeiy pr^^termined by ttw ori#nd{ 
TV Culhj^ ulij«2t«ve, lumdy . MiuailMty ^ stuiksnt lo cumpiett? ttie entire two-year «iilege f^ogram Icndins to tim 
Assop^e in Arts i^ertiftmi' without fe^yinghis hofm, by combining teti^icni vtew^ir^^ am^e$porKiariC9 
itudy. Geit^ai ahicatlan L-ourses -e fl., tSfaturai Sci^i^^ Hun^nittes, Sodat Si^na^-f^H^ of alt sURtei^ who 
are ^raduat^ from the City Ckiii^gi^ of C^cs^ have bem mt^NEt at regular tntetYs^ ^ tfuft a vH^r taking aU 
his college courses chi TV &n comptei^ ttvs tvm-year f^nogrimiG^ng tci the Ass^ate's tfcigrea within three 
y«ars or so. Two i^ter«il courses aru of f tMiKi durmg each sernester of the »»teinie year. Ikit« as aln^y kuficafed, 
most students <aiinfjt6te ofily four uf f«vt» counes on TV- Very few comidei^ an entire two-year progriHtt. 

Other of fi^rinys jre "dectfves,** madtf up of courses not ra^uired of all students seeking the Associates 
certificate. The preferences and academic <|oals of TV Cc^te^ c^^x studenft-^ nwte kfKimfi by th^r respor^ 
to ifuesttotinaires as w«Ht as liy otht?f surveys-have remained fairly constant over the fmt twelve y^rs. Elective 
(xairses schedutc^i in ctusier s of 'liands/' e.g., nythematics an^ ^iem^ bmir^s se^et^^ial skiUSg geanerat 
cuilurat sublets. 

Snc8 the early days, 40% of TV Coile^ credit sti^tentft have reguAarty ^ifsress&d a ctotre to enter feacto^- 
training curricula. Accordingly, courses related to their interests have been sdieduted each year-^for examf^e, 
introductory courses in educatimi (e.g., i^i{oa3phy of ^ucati^« history of edi^ti<Ki| and psydiolqiy. These 
courses, especially when pr<asented over i^ short eiy^t-week suntmer term^ ^o attiwt teachers looking for in*service 
training opportunities. For example, a Summer 1988 ^jrvey disposed that of ttie 45S students mtroiled in courses 
in Philosophy of Education and Educational Measuren^nt and Evaluation^ 77 already h^ bachelor's (fagree$ and 
10 heW master's degn?es; 6 held regular putHic sdK>oi certificate^ 26 provisiimdi certificates; 47 tod tau^t during 
the precedir^ year in Chicago-area pubUc and ^ivate schools; S2 had enrolled in oite or both coti^ses to fulfill 
requirements for re^lar public school certification. 

iE is noteworthy that TV College experience in this respect forecast initial Open University experience in 
England, A siitabte portion of the first-year's Qf^ Univs^ity enroltmeint ¥^ o^mpri^ of tei^im se^ii^ fuller 
certification and technicians seeking high^ credential ing. It is also noteworthy that there is now a surplus of 
leadws in the Chicago are<i, a factor to be considered in future TV Coltega fm^^mlng. 

Only recently TV College has t^un to ^ift its oirricula emi^T^^s, pro*npt«f a number of factcm, Brst, 
the State of Illinois requires that the public comnumity roltege devote a large sxirtion of its total oirricuium to 
technicai-occupatior^al prograrr^. Secon<i the TV College staff, ahd^ys iMrmmifortM^ ^b^ut its f^'c^^nting 
holding appeal only for the relatively narrow audience capaUe of oimpietif^ ccrilega and university a>4«ses, is 
ea^r to present series which hold ap^l and value for viewm wfK>^ interests ami o^xacities are rK>t a<»}emic. 
What this means, in short, is that more introductory courses of vocaticmat nature are now being prockfced for open 
broadcast. If and when an ITFS system is avaitable-and if and when CATV systems 9te ir^alled within the 
Chicago metropolitan area -vocational courses of hi^ly sp€^idli2^ natt^, aimed at groups mc^ too small to be 
served by open VHF or UHF broadcast, can also be presented. 

There have been nolireiiWe shifts in audience preferences within the past iweral year$. Some of these shifts 
reflect changing studeru tastes of conventior^t campuses-for example, the decline of imer^ in foreign lan^ge 
study. 

Audience imerest is persisting in cultural sutjjects. Viewers still erroll in lar|^ nurr>t^rs inert, mu^c theory 
and appreciation^ philosopHy , and history course. This is not surf^'ising in a maj<^ urbmi area in which are con- 
centrated lar^ numiiers of iieople who display the traits of wfiat the studies identify as the ETV acKtiertce. 

In recent ynars, t(K>, riiorr proqfammuxj has been ftevotefl to "adult education/' Usi^lly tfiese ventures are 
supported by funduiq from utitsid^? ';cnirf:fs. Spwiai series havtf been done in areas like Real Estate and Art 
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A|if>f e4!i<it{<in As yt't, with llu» **xcijp<*t^i of a &tf itfs iti oitisumer i^m, Mttlt? has tieen cluiw in imic "aopit\q" 
likMs lilt?! aciy^ <*tc. It k ex^?cttil once an ITFS system is a reality-thai ofterir^ in a variety of ifeiitry levd 

Ut stimnui v, suwMi<*<ni y*?jrs «f OMp^ii^mse, and a steady tlc»^ of m^orm^ton derived frtim stiulent 
4|iie^tHMinj|}tH», iuv i^uiU^ m scti^ctinq iwul sd^luitng course. By rotating OHtrses carefully arKi tK)t tvpteiing 
tlH?ni at tmi frej|U4?nt utttfrvaHi, a httjh tevol of credit and not far^^it efwc^ents h% l>^n matntaned^ 

As has alfeady tntun indicati^l, TV Cotiege is now ^k«f^ to hro^n its i^ts aiKl cidtivate audiences with 
mmh tff ffefvnt Umn those id stittltHits seeking the d^ce. As a result, the yt^ars to come wilt ^ mof« 
|ir4if|uinin^ik| witli an ufifnt^iakuhly ocoupati(Miai and fsotfrnnuntty service ^npha^* 

^ Refises of C^if^ Cc>uisos firov^t to be! effective m adiieirins their c^^iv^ are repfayfri as often s& 
f hft»i» i#i tiHir tmiiH ovtff a ^H!Mmi of ftve or six yii^s. No course, howem, is replayed until it been thoriHis^iy 
ivviuweti an*l inlited tiy it<j teacher, with a viafli to inwoving ar^ updating. Usually the teacher will remake erne or 
two f>f thif progiams ^n their entirrfty and remake portiiHis of other i^oyrams. 

Chw thinfl shiHitd tie notcnl with resp^t to re|]4avs of oiurses: they must be schediiled witit sufficient time 
hetweisn showim|s as to maintain th*j level of cr«lit enrottment necessary to keep costs ^voiiMaie. The level of 
audiLMict! for ecnnomicai upHration is tnit constant. Usually 3 cr^t course cinnot be of ^ed rmire than once 
every ollwr yuar. 

Si>me TV Colle^ courses are rele^ed for rei^^ outside Chk:ago. Sin^ 1^6, the Great Rains National 
Instruct junal Television L it)rdry of the University of Net^aska m Liircoln, htebraska, h^ handled tenlals of couises. 
CiHifsi»s are avoflabli? in vKleotape or videocassette. Community collet outside Chicago, but i/rtthm tbe Cli^nnet 
1 ! siijfTai ai ea, are also free to accept enrollments in OHtrs^ as tfiey are bro^toa^, provtcted ttny supply local 
cotinftnatfon. Bfyinning m Fall 1973, TV College courses will be made avaiyi>ie to all Mlinois fHifolic community 
c<>llef^*s without retital chanjes, 

Interiitstftutiotiat use of courses has been slow to develop in ttw U.S.A., ctesptte nruidi talk ^>out it, CWy 
within tti«? ^ht^i year hav4i t>een signs that shared use is on the inarease. No doubt the new emphasis on c^n learning 
systems, i^runif^cd tiy the success of Britain's Open University, has had somethir^ to do with this. On the whcrfe, 
howev^jr, fdCiilty resrstance, dee^^y rcKJted feelings about institutional autonomy, and the generally unimaginative 
prcHluction qtiahty of much instructional television have impeded both mtertnstitutional me% of televised material 
and nifennstitutioital pfannmq and (tevel(H7menl, 

Vw Components Anyone administering an mstructional television service soon learns that a telecast, no 
matter how well structured or imaginatively produced, cannot carry tfre entire teaching burcten. Printed materials 
and apiw-oprrate face to- face activities are also essential if a high level of student interest is to be maintained aod 
student success insured. 

F very TV Cnfte^^ course is acmmi^nied by a syllitf)us called a StiKly Guide, the content aiKi sospe of which 
•dries with the nature of course and the fjreference of the television teacher. Every ^ide, however, does contain a 
stJtem^-nt of course objectives, a detaileti course outline, and precise initrucfiorH as to rti^ings ard written projects. 

OvtT th*^ y^?ars ih?» TV Cofte<^e staff has discovered that the judicious emptoyme<it of self-scoring and 
f>rot|r timed learnifiq matenals, based on either linear or branching metlKKls, reduce passivity on the part of the 
student, and, m fact, incniases the percentage of students completing courses. When editing series ptior to 
f*-pl«iV*». tf?3chm ^^nmuf.ii^Hi tn pr*?par«; such mafenafs. Studies have shown that thcs^ materials im|Kov*? the 
uiXif of sttideni f*f?tnfioii 
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t!^ regular iy emt^tt^i stutfc.'m earns his &miii throi^ <me of ttm coHc^ of tfw CSty Cotle^ syn£fm« not 
ttirot]^ TV O^^lege* rejHirts to a TV OoHegp cam^ center for thrw examinalfms* The ftnt two-iw 
•^mi€tli!f m'*- oiv cocH m^e hmir lurm. The third-or ''final is two hours km§. Face-to-lice cofifmni» wtth 
sttMeot^ aft? scht?<hiU^il sevt^i *tJ timi^ «*^,h tinm. Thfi tifachtfr or coottlmator of a teAevistoo course scttedutes 
himseif Mr fetephofi« confi!rerK:es >>vt^y week, (tt c^taio f^mrses-a.g.^ ^ofth^v:!^^ tytiifig« for^spi ipOgimge--tfi^ 
t^teyiMHi studeni i% requirtnl to titteiui 0rH:;ampus ddss saisiortt s& irany as sev^ or eight tinm a tettn. 

Most courses, too* entati a si^jniticai ft oirresporideiice cinnpooent-that sluKkms imil wHien woric to 
Mfyfi%ion instructors or other toacfiers (nalied '^Section Teaehws^K Th? i^iectiv^ ami nature of the (mifects are 
&p«ciht!c1 m the Study Guitlc. 

The Facirity IVk»st tiMi htft s %:mm to TV Cc^lu^ from classrooms of the City Q^teges of Oiicaso. tt can foe 
orctued Wmt this hds tjeon one of Iho shortcomings of the |:M^ogrdm« Tte Ford FoumiatitiMi consuttar^i when TV 
OoUegi! ptanni ng w^s b^rt^ly utkk'rw.sy in 19S6. sug^^ed that cetebr^ed st^Krtars and ac^ftemic persoi^lties frtmi 
Mkiwestet n umversttit,^ line i ecrutted to present courses. 

On siTwraJ occasions, professors frnm outside the o^ik^ have been recruited to taiK^ courseS'-in Astronomy 
ami Anmtcdn History, for eKoinpH?. The ex^^itoe \m been sormwhat tSs^H>£>in^n^ Vim only time tlfe «nploy- 
ment of an outside teacher has been siiccessful is ^ten his oKvn Imtitution m^Nle the tetevisic^ ocMSe iti qitt^ion 
one of thetr own offerings and enroil^ students on their own campt^s. TWs is not meant to inr^dy that outstand- 
ing professional talent sttuuhi not be recrt^tetf wfterever it is found. Ceifainty, ootfegc^ und^tafcii^ tt^ production 
^ of courses on regional or cooperative bas^ wtil want to reo^uit* if possible, scholars and teadiers with reputations 
that go beyond their home cami^ise^. > ^ 

TV College has only seldom employed instructional teams to as^t ^e ti^eviticm instructor pr^are his series. 
An important reason for this was the early comnntment to produce a telec^t that is essentially a visually ^^4«am»d, 
carefully rehearsed cl^sroom performance, a commitn^m dictated-and stiH dictated in partly Hmited futttling 
ami the necessity of presenting a ran^ of ctMirses eadi term. The irMructiomri ^am afi^M'O^, if it is to be 
effective, must have a teteyistofi f^oduf^r member with strong acmiemic ore<tenti^-a oo^mrKKlity rK»t reacUty 
avail^le on this side of the Atlantic. Carried to \X% ic^cai outcome, it would mem to fe^ to the employment of 
professional actors or broadcasier rather than teachers, as course presenters--d f»^c^re which, desiratrfe tfKMigh 
it may be urxitir certain circuntstances, is l^yoitd the n^ans of a sif^e ir^titutiwi* 

TV Colleger lacks, it mei\ harijiy tie sakj, the resource of money ^hI p^sonnel required for thonHighiy 
"mediated'' instruction. Bor t^vef^ if resourois were availabfe, it is questicm^Me that televised instmetion internied 
to impart cognitive learning arnl complex skills can fcw divorce entirely from the trailer ''figure " TTwe is a still 
unexamineil premise which has . taken root in the educattcm^ bromicast worid^-and par^ of the ai^derrac 
community —which holds that the fast moving, high-visual fKodmrtion terftniques effective in irnji^ng the 
norhcriticat frame of mind needed for the willing reception of the ^i^tttinf message-or cafKrfdrte of comn«r«feer- 
ing the attention of the pre school and {umiile mind-i^n be ac&^ad wfwriesate by allege iml i^^versity broad- 
caters. It is conceivaiilc that close analysis and orckriy step-by-step i^i&sentdtion require sidxHied and undistrusive 
production methods-a tailoring nf rrwans to ends. AH this, of course, is a for the res^cher^ 

It is true, of courstf, that the talent ^x>nd in most imtitutions, espeaalty in two-ywr colleg»« is relath^ 
small. But by careful recruiting aini maintenance of favorat^e working conditions— e,g.^ ^eqtate tirro for prepara- 
tion and studio presentation, a full Summer for preliminary ^^^ation aiKl or^lzati(^ of study materia ^ 
& full semester with no duties other than studio recording-TV Cdllege h«i nonaged to recruit a nwter of imtnArtors 
from City CoKeges of Chicai^) campuses who have learned to te^ effectively on television. 
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TV Cotter Stuifonti and Ttmt Pe^&nmtn^ 

TV Oitu qr < lassifu'S a^^hi stinltHits R}lfows: a) Homuview«ris; M TV Ci>iiairri?m students; c\ TV in dasi^ 

Htwm^iwers an? «:ral*t srutienfs who watch d^ses at htwne or off Cannes ami hm? no rcgutiH* contact with 
c^ittiMm af:t<vilii*&. They make ufi 75 u» 80% of the total crudit enroHfneni e^ery loriti- 

TV QftKH^H$ts dtu HtotliMtts who takti courst^s in tl^ d^rMm wNte unnitted In TV cout^. THev wiay watdi 
o\mn itrfmisii* 1uuin\ or un t:ant|His on vt^fsutuim m yicteocanette drcutt. ffnsy fwkur up 20 to 2^ of the 
/ toM «*tnuHfni*til fjv^iry h*im. <^ . " - 

TV m tiim viewt^ ^rt* sindtrnts of mjrmal coite^ aye view tetecasts in in tf^ das!»w)m. In the 

1*4^1, tlif?v vwatrtit^l courstjs as they wwe txt^icdst ofi th€ air. Now^da^ It iS more «»nn»min for thwi to watt^i on 
iluiiinl i iujMtt broaik:.i&t, with ^vograim tramferr^d to one-indi videotape mo4^ or vkfeooais^^. These sliicteitts 
niit?t with a cLi^vfuoni tt»jchor, who inay or may ttot be the tdewisicm teiHCher^ once a week for ^ftkin«>l imtruc- 
t«of^ kintt of ifistrutrtinn has iHjvtn ftmirtsl^ on campuses, mdicaied tidow, research ^k>ws condi^ivdy, 
ftowevi't , that iuuii?!<?i:t*jiJ $tutltMit$ taking TV coors^ in the dd^srcKim nam have supplefmntat d^^sroom mstrtictkm 
if fh^y iire to match rtii? fH^rfonnance of ailult homevtewers. 

Not fnr<:mfit %%iHk*nxs do not take eK^inattom, nor do they submit writt^ wwk. For a nominal ro^^tra* 
lion ffM? (SI ,00} lb<«v an? stmt course study guktes. No official records s^e km^t on them. 

As niu^t if x|H'ci<?<1. the stuiients who enroll d$ nonH:r<^t viei^s trasses a rdativety high kwd of e<luca* 
tHMiaf t>*idt yrijutxl, M^iny have wniveiiity de^e<^. They are tt>e people one wmild expect to watdrers of educa- 
ttonaf TV, Tli<?y read hooks ami serious n^igazines, follow community affairs^ attervi conc^ts and the theater- 

Stm1ef}t Acmiths Visitors to TV Cdlege alvi^ys ask about tf^ mKhanic$ of planrung contrc^ling 
ai:ttviti^ hir thouwmis of tifudents within seventy-five mik! area. The |:^esent system, it goes without saying, was 
devdofHnl only afier^mudi trial and errm^, 

Shio? nnnt sittMhfins an? removHl from City Oiheges campuses, folders listing courses to t«? offered, the topics 
of f)<tM|f^4im, t*»xtlH>oks r<HUHr<Hf, !he broailrast schedules, and the places and tim^ of registration, miKt l>e rr^iled 
wit each tt?rfn. Somt! 40 ti) 50 thousand copies are maileti out each term, ami about 10,000 are distributed to pii>lic 
hhfanes, ^chooJs a^nf public dtjencies. 

St4H<enf s trnroll in dW scvi*n coHei^ of tf>e City Collets of Chica^j. Physically hai>diaif)peil students may 
r<»i|<stef i>y mail, as w*»if as students who have completed several courses on television. A hundred to two hundred 
sitHlents arr i^nrolled in penal institutions. 

WImm! stii<lfu>ts ri!uisti'r, thi?y <»tect a TV College Center to which they report for examinatioits and confer- 
ences. Tho four crenters are lin:Et!t?cl so that stutlents livini? in all parts of the city and its suburt^s Can reach them 

vastly. 

All crmht sfudinus rin^ givrnr a copy of what is called a Cr^it Bulletin at the time of enrollment. This 
intlMiri rofit.iins %hv data's and timi*s of exammations, the telephone numtier of television teachers and times 
when ifiey am be n^twrheti. as w<fll ix^ other n^essary information. 

A^ tifriMdy HultrdtiHl, %tiidOrifs taki^ three examinations in each course: two while the course is in i^ogiess 
♦ind oiH* at th<» r'fvf . ffu? Cr**fht 8uHi»ttn indic*i!es the dates ami times of examinatiot>$. Most eKamiruiticMis ar^ of 
th»; obj( ctiwf, muhipU» f huic* Vcwirty which can \m scoriKl by a machine. All thae examinations are "item-analyml/' 
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ttiat is, amfyml with a vnw to dettRf nWninf| w^at (X*rcenta9es of ttii<fem$ owrect res|K>mei to it^&im. These 
aiMlvi^es 4re Stiitt to l«$aditfrs to help ihtmi improve eicaniinattGHfis* {k shcKitd be mte4^ too* that sotm teachers 
r€t|uire essays as pan of eHam^nactofts.) 

As indi<»l€ii H^arHer, teacht?r$ sittieiluie themselves for tvw> hour4c^ tetephcj^ie eonterenees ji»ch is^eek^ durifig 
whicH tifties students may ti^tt^phofft? t<i seek hetp or make eomments* They also sdiedute ap«i eonferen^ at 
nMtich tf they mc^ with thnir students. 

TV Coilege makes H)eoiat pnv^istofi for hamftcapped ^udents, Stttdetite coAfii^ to their hemes or hospitats 
by dtness ami physical ikfects af« attmnmtered exainftiaWcffHi tiy proctci^-t^eigymeiii m^^^ cht sodal vifoilcers* 
Students who are inmates of the thri^ penal institutions TV CoHe^ serves are administered examinattons by 
memlH^rs of the prison Hilucauonal staff. 

Over tht! ^last rio/en yi^jrs, Chic«^'$ TV Cottei^ has conduf^ ^refiri itudi«^ ol Sttident p^formance ami 
sttartsii its tTK|>eriences lo seimrat published t«^rts« Early Interhrt f4$i:)^rt$, FkuU ff^pcn^B Iftiw? Ymr Cxperifmnt 
ami Qtfc^^s TV Cs>R^, summarl^ed TV mnivfties from igS6 throtq^ 1359, 

The first -nametl reinut, ain^ at re^refws ai^ ^^^c^li^, presented ctot^lt^ stat^^ df^tlv^ of the 
performance of students Uku)g cOMrses on o(^n*i:ircuit tetevision. Ourinf ^ first yeair irf the trials exp^tnentai 
atlentkin was directed to comparni^ Iht* achtetfement of junior t^H^ stiidents viewkig TV ami studying at home 
iMtth that of stiidents takmif the same courses In ocmventiCKtal cfai^oon^* Tf¥s perf^mar^ of sttKfen^ vwatehrng 
TV lessons on campus ar«d receiving fotlow*up d^sraoth Instrtictton was also studied. During the seccmd year, 
careful studies were made of the performance of the 'TV-at-hcmte" sttKi^ and that of ttie eveni^ sttidents of 
mmparable a^ and motivation taking the same course in convemion^ cla!»». To oontrd the te^^r variable, 
whe»)ever possible, the TV teacher was ;^igm!d to the orH^mfH^ ntroi group. TT^ third ye^ saw exp^mental 
activity centering arpynd the use of TV series as direct imtructir<i fw um^efect^ Jutibr college students of normal 
collet^ a^< Comparisons wert! n^de tietween 'TV-in-ds^^ and oonventionaliy ^u#f corses. 

The authors of the fsml R^>ort came to th& fottowing g^erai {xmdtfilon: 

When evaluated by the techniitui^ of meamir^rmnt and analysis us^ in this experiment, 
television imtructton is a thorou^ily effKtive m^ms of extending ccMeg^ oi^KMtumties 
to at -Home students in alt suh^ct'^reas explored in the prefect (p. 

Among more specific condtisions of interest were the following: 1 ) the at-home TV stiKlent, typkally a hi^y 
motivated mature aduk, tends to outperform his counterpart t^ing evening tmir^ (m campus, and 2) unseJected 
stuiJents of normal collet a<ie watdiing a TV course in cl^s will not fmfcmii satlsfanorily ufriess they are 
f^tmided foliow^up ctassroom instruction. 

The consistently hic^ level of perfwrnance of tfie TV at-honw sttnient does rm\ si^ial any sup^'iCN'ity inher- 
ing in televised instruction. We must look at the stud^t. A hcmievtewir^ sttKtent ^tected at random woukj 
f;»t>t>abfy be a ^year-old wonitin ^ ?B% of tl^ credit aiHltent^ is finale. TT^ diarares exisllem that she is 
married, with a homa and chilciren to lcK)k aft»r. Her hi^ sdxKH reo^ wffi ^>od. &\e pr<rf^^y finbhed in the 
upperhalf of her class. Siinre she is kept busy as housewife and mottw of smatl diikben, can take only one 
w two courses at a time. Probably she had no opportunity to go to ct^le^e in^fwe nrwri^, or if ^e did, was 
able to complf.'te no ntore than d year. She is interested in maktr^ a career fof herself ou^de the hcmw-'eftfver to 
experience a sense of p«irsonat fuifitlment or to add to the family income. TV Cdilege courses start her on her 
fourrwy. Later on, then circunwtances are r jc^t, she will trar«fer her credits earned via TV to another colie^ and 
eiuoi* in c^fiventional course's. 

Data havi' been (pthered on TV CofliHfe students ovar the years. Several thcH'ou^ studies wme made in the 
md igSCs, ificludinq a Ph.D. dis^titatiod. Thrse studies tended to confirm the view of tl^ TV College credit 
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stiiitom as possess«ttg a comf rcHMtt* of lower mtddle clss^ traiis-with a strmig drive for upitfard maUHly , im<¥est 
111 hMrnkiu fqr the %ake of pi'r^mal advan<>?m -v , m*\ Wast TV Ccrtte^ credit siitdenU hme been dustered in 
hmci miildli* dass arnl inut^Ni? diss nwntminitles ^ the metn^Htaaar^a; The gr^atesit incMent^of failure In 
I V CtilktH^ iKt fof maiKJt? UMiHHi tfi i^luiieius who five in iiww dly afiid tUsjkfy£^lat|ed wrtp^ of tfte city* For thit 
ntaf tin. cwoUments from tfH!so at eas make up filite more than b% of lotat en<oilmenl. 

WilHin the P»«^t ^-w yeafs, ttowever, jKjpuJation movement out of the cfiy has ^xetet^ted. Oedit enroltmenis 
ftoni siiburt^an ariifas haW) bt'e^i ^c^luml ^arpiy because suburf^n areas now suf^drtit^ puMic aimmumty - 
colkuje^ ari; no Uwtipur tetniirwi to jiay fuilkin drarg^ of fiDsidenis who emtill in the bhic^ system. (As Imtic^eif, 
t V Colli*!}!* t$ tuition Ire*? uniy f oi CNcatfO residents J TV Cottc^ itself is ncsw attempting to reach the disadvan^ 
Ijiitnl stfipmHtts of the city's popuiation. 

•ti - ,' 

Th(»fe H a real itenl ptesentiy for tnfiM^nF^ttmi alKiut tfte kinds of TV Ccrfteqe stmlcnts. Skm^ of the data 
l^n in^ <(at h wnf throui^ th?2 tww St<M^ UnUmi^ As ^ of th^ Activity, videocai^tes of TV Co(i^ 

tcHUirt?^ fiii»tfd in Chic^<yi Pubitc Library study centers. Sev^at ^aduate stucten^ in Educaticanai Measurement 
ami EvutiMtujn as$i4infetl to TV Colte^ and the Leaning ResoiA^ces Laboratory prefiaring an imtrunwm 

10 i^, dfMfihuttHi to Sti/r/K Uniimited students who enrotl in these course, tts purpc^, besides gathering data as to. 
ayf- iHl.irattOfMi had^tjround. etc., is to determine what the elements make for failure in off ^mpus independerft 
\tudy. 

Costs ' 

Anyont! aci|uaint<^^ wth open<ircuit television prochicticm knom that it does not come at cutHrate prtc^. 
Jniuines dliout cost takes '»t»v**raJ fornn: e.g., how miKrfi does it o^t to produce a singte <:xiurse? how do TV 
Coitefie costs m student compare with onncatninis costs per student? What are the categoric of cost? 

Overall TV CoHege costs cannot tw discussed until the fa^urs entering into the calculations are identified. 
Signified^ costs include those tcv } ) studio production and trammisston as est^tished by an anrmat contract with 
ihe televisioti studi^; 2) teacht^i , production^ administrative and der ical lakr^s; 3) reference arni researdi 
activities, graphic work, cm-^ite filming; A) preparation and printing c stiKfy g* ides, f^cmtotionai materials, imuI 
eKanHuatKins, 5) supportive (mtrmrticnal activities— folk>w-up dassrmKn instructi<m, conferenc^^ omnsdif^, etc.; 
6) <fKlf ri>ct instructional services^ such as exar iination proctoring, spedal r^fgistration activities, etc. ^t induded are 
atstifor on<:;3mpus library Services, regular registration sei'vices, heating ar^l lighting of ct^^ooms arwi labcKat(Ktes 
UiitfiJ by TV College stu<lents - services avaitcrf}te from 8:(K} a.m. until 10:(X) p*m. on City Colleges campuses v^ether 
TV students are present to use them or not. 

&nc(^ atiout 1966, the total ar^nuai amount t>udgeted for TV Coll^ ^ivitu.. exchtsive of foundation or 
federal and state grams earmarked for special fM-ojects— has been between S800,(H}0 and SSCW^CKX). Each year some 
S3^),000 goes for studio o(^raticff>s, about 5^/b,0(K} fcK teacher and indirect instructional salaries, and arK)tf^*r 
$250,(XX) for staff satariirs videotape and equipmenfi purchase, and ot^t^ad. 

The cost of ^>roducing a thirty-pro^^ M&^ninute^ series is currently softw $60«{KX), of which S35,(MM) i^s 
into stuilio production arnl purchase of videotape stock; $15 to S16,000, on an aver^, for teadw's salary; about 
$7,500 for the st?rv!ces of a produLer's assistant, ^afirfifc artist, and scene (Signer; and anotler S2,5(K) for adminis- 
tratfv^ and clerical SAip^>rt. Transmission costs for twice repeated on-th^air t^oadcast of an entire series add up to 
aiK)ther S7,000. Added to this are the expenses of printing sei^rat thousand coiwse study ^ides and examinations— 
another SI.^X) to S2,0^. Thus the tota< outlay for mitial production arrl on the-air presentation is some S70,0(X}. 
It mu^t l>e borne in mind that an effective course can Imj presented on ofwn circuit, with only minor editifig and 
ufRtd^ing, as many as four or five times over a stx or seven-year span. 

A cruci:! qiKJstion, hnwcvfr. is the cost of TV CoHege instruction on the basis of student a edit hours generated. 
Pui inure simply, how do TV College costs ptr aedit hour compare with those of conventional instruction? The 
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a^age cost f^^^credit-^hour of imf ruction in the City Colff^ps of Chicago is $&0.(M>. (A fuH-tbtie student earns 
fifteen cf^iit hours tn a sem^ter.) 8y enrirfting the 4K}ui¥<itent of SX) to SOT fult-tliiW nuctents each semester 
^iKM^ft 20<X) individuats} ami fM-eserttlng s©<^ courtes« of wtiic^ only ttwee me beli^ pi«^ite^ tor tiie time, 
TV GoHege can keep tt% p^ creciit hour cijst at irtimtt fcHrty f Ive doliars. Thoi^mfe of tHin^<»^tt ^ewet^ am 



CONCLUSION 

The fomjoing r em^ks hai^ presented in outttm what are ttHf feirtm^es tiNM d)an»^ize a t^vtlskm-based 
open tearnin^ system with, perhaps, the ioni^t history of conthmofis Ofmration in ttm nation. Tte it one 
that does rKit tend itseff to duplication everywhere. !t tm atiAays bean hHturate inhaling mthiidlertii^e*^ 
imvmveri ng administrative suppc»t. It \mB also been fortur^te m having an extef^w broaden ^miute tm a fvui^r 
puttie tetevisfon chmnel. Above all^ it could not retnain effective on a cost baii% VM10 it not for the lar^ iflban 
population it reacN^s. 

To continue to provide meaningful s^vic^. it trmt stay ebrem of diai^ in its oomnmnHty and seek out hew 
audiences. 
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VIDEOTAPE APFLtCATIOtSiS IN ENGINEERING EDUCATION 

Lion«i Baklwtn 
Dean, CoHe^ of Engineering 
Coiorado State University 



OFF CAHtfm GRADUATE PROGRANS \ 

Jlw rd$>ici fidi:*' of tiMidV'ti techrtotogy and its {ndustriaf dfMicatiCHi had a yeat impact on engifmriftg 
iHl4K:anof^ m Utiited States, ftior to 19§0, the four-year tecMor's deyw was t»^ of ct^uice for 90 

purc«?nt of thi» qr^nfujies in engineeruig. Today, we graduate et^ettf iatly the same total size, but over a third of 
tho stiAl^Hits recHitft? qi^duate cfDgrc*es. Th*» a<<<iitiona( stmJy m6 speciati/aticKT which the degree in en^neering 
(iffers has lead sonti? ^?<li«cati(>fial Utadeis to argiie that professionat practice in the near fu^re wift require it as a 
|.>rert»<|UJSftt?. ^ . " 

Appr uxinwtfly half tif the M.S. <k?grees dwartted in en^r^'ring today are eat^rred liy part-time stuctents. At 
least sevt^nteen c olleger of tfngineenng in the U.S. now use television to extend high quahty, advance de^ee 
education to stuttents «ni|>loytHt at locations renuite from established camfHi^. Time are on^ing, diverse learning 
systetns whtdi 1 t!sttmate reach over 1 5,000 students anni^tly in at least 7^ separate course study g^otips.^ Eadi 
year sev»*f al addt tumal roHeges of en^neerifK| bring similar progp'ams on Utw. 

Employers of Engineers are (^neraiiy res^KXisive, occasionatly enthusiastic^ in making available the m i^ant 
fa<:iiities for vnteo nistruction. My expenence m managing and enlarging the Colorado State Universir/ program 
lK*raitek the findings refjor ted in a 1969 NatfCHial Science Foundation report on ^'Continuing Education for R & 0 
C4«Kse^" {2K The f»mf>lnyer5 typically vtew the new vtdep tmed program as an £Kitlition to ffieir existing tuition 
r?»fund program ami imforiutiately, tliey tend to manage the new learning situation £S 3sn additional fringe i>enefit 
fur thtrir professional staff. There are few serious professional deveiofKnent proyams, but on the other t^3ff}ii, most 
tjngmeers da feel enc<Hira<fed to p*Htic«pate. Oily about one-third of the partiqi(^ts are oriented toward dei^ee 
work; these teful to t>e the younger staff memliers. The majority of the participants prefer the university credit 
< iHirses fof if i ilepth (earning of the fundamentals of a subject ^ they seek job-relatad diversification of their educa- 
tuiit latlw than a tk-t^ 'exit's worth nothing that the 19^ N.S.F. survey of 17 targe R & C facilities foumj that the 
university credit course was tfie preferred mocfe by employe for cwitinuing their education. The professor's 
kfctiire was a m<in? ^wpular fearnmg sitiiation than employer siX)nsor«i non-credit courses, short intensive courses off- 
site, professional meetings and in-plant seminars (2). Some media experts are incliried to arg^ that this preference is 
due io many yeaf s of coinlitfcming. Nonetheless, this prefererKe docs «sist the video prc^amming in this special 
e<hicat»onal context. The fion participant?; in the VKieo-hased systems are generally older; this import * ;ssue will be 
discuisseil later. 

V deo defivf?? y is now t^nahlnig mature, well motivated groups of practicing engineers and otfier professronals 
to particulate tn t^oiiuafB cretld courses at their place of work. These courts are regularly scheduled offering on 
campus attenctetJ by full tmi^* <;tiidenis. Ttie dasses are held m s^^lally equip^d studio-classrooms ^ that not 
otUy the lectures tmt also the stuttent <|uesf ^ons and discussions are transmitted. Appendix A details the (acildies 
uMJd at Coloraiin State University which are typical of the classroom settings emfrfoyrrd in aM fUograms of this tyi>e. 

*lhj«^ «»)$t(fT»«i«^' ( nj'h?, Of tier (T^^i^fUiiif tMiiutyt^r% q»v«ft hura lo g^ve ^ome isied of n The f<rst if^tioniir «tifvey nf 
ffiftw* itfatp^n% i« fMYj^ iin*f*<f wv,iv itntUir th« .}iit,iatum oi ^ special task cotunMtf'fp an "Th« Coir f 0«*r:f ivivrHm of CotiT»nMinf| f- 
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Atthoui^ the stuillc^ciast^riKmi siiudtioits are similar^ (here are a v^iety ot si^sd {Mivery sysfeim errqiloYed 
to iink th« iiiifmtrial facilities to the caitifNjs. The first maiw system (1964) at *e Uniwnwty of Fkrida im$rtoiM 
tWGhwav. point to point microvvave «ii4tic}i wm In^sitd from the tetefrtimte ccimp»iy« to fink the main canqnis to 
Viiim^ exienstuiv centers in cetitrat Ffondi* (3}. tn 1^9. St^lcNrd thiivenfty begw iM»vin« IfHJt^t etes^ticmis in 
the San f rancfsco ftay Area witfi a fcHir chanm»i f TFS (tnstmctionat Tefevisii^ Fixed Service) $y!&t&m wrhich 
featureit FM-talki^kcaf^tiiy (4). Protect Cok^ado SURGE (State University Resourw for Gradtmie Edwation) 
i^as the first ( »967 ) to emi rtoy ctnirrier carrier videot^ ^ a delivery system (5). Some of the newer systems, 
si^ as the Okiahoma System (€K no%v enipioy interconnected ccN^nbina^cms ot iKrim^to-lloini miei^^waye for Utfig 
hops, ITFS for <»striUtitton in pofAilation center videotape for rmiote ar^ smd make*up of missed bnoad- 
casts. A tist of the engineering coirege {^o^ams and the fKimary delivery sys»!ms munr c^ratlr«| is incluCted 
Tabte I. Active debates ccfuc^nin^) the relative ma it of th^ n^^nrrisskm sdiemes ones n^g^ on the ifnt>ortis»iGe 
of interactive tatkback during the tietween tfte remote student the prof^c^'^ ver^ die gfemt sd^Kiutlitg 
flexibility of videotape systems. This has subsided and discissions now center on a ^timai an^sos of popuiatic»i 
ginupin^ in a gi^i regeun and ^u^nomic analysis of how to serve them witfi vartois ptif^ tef^molc^es. I want 
to come to this point iater. 

TABLE I 

EfifineeHtis GhnRluate Progmms CNf«red Off ^Cm^nms by Trtevi^m* 



DEUVErtY Mooe 



G€NESYS - Unwersity of Ftwida 
TAGEB - Southern Methodist 
SURGE - Colorado State University 
Stanford University 
University of Tennessee 
towa State University 
University of Color^io 
University of Minnesota 
University of Michigan 



Pt-tchPi MfC^owave (tea^) 
Pt-tO^Pt Mk^-owa^ (owned) 
Videotape 

fTFSmdFMTaikbadc 

Videotape 

Videotaf^ 

Videotape 

ITFS 

Pt-to-Pt Mkyowave (t^^) amJ ITFS 



Uf^verstty of Southern Catifmnia 
OA.iahoma LT.V. Network 
Univffrsity of Catifornia at Davis 
University of Pennsylvania 
Case-Western Reserve University 
SUNY ~ Buffalo 
University ot An/ona 
Cornell University 



tTFS 

Pt-toPt Miorowave and ITFS 

Pt-to-Pt Mit^owave 

tTFS 

ITFS 

ITFS 

Videotai^ 
Video taf:» 



•Tfwfs m/iY l>e a f»w orfus««ons or new programs iMifth nwre mr«ft9ct tHit thft foprewntn the mitfiwi' bwt kncmrl0clK0. 
Lm «s «pprOff {m«t&ly chronoiogK^aL 



A brief ctescription of the SU RGE fwoi|fam v^itl illustrate many features ojmnwi to these outreadv programs. 
The course work is delivered to industries in the form of videotap«f ci^ sessicKis with su):^rting written materials^ 
produced for classes on the CSU campus. Every course in this program is a re^larly scheduled offering on cantpus 
attemled by fuM-time studei^ts. The SURGE classes are held in specially ec^ipped studio*d^roon^ that not only 
the lectures but also the student questions and discussions are reoorcted vitleot^. After the tapes are made, they 
are packaged with class materials, assiyiments and eicaminations m6 carried by a commercial delivery s^vice to 
each of the industrial and (jov«rnment locations. The class sessions are vrevwd on a regularly scheduled bash by the 
of f campus students. Tho of f campus classes usually view cliss presentation two days following the on-cantpus 
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ciass: ovw m i^tmn of ttwse off catties ^j&icmts are during fegufar woricins houfs. Tap«, however, may be 
astatiied by ^i? iixhintrv tfwt any t^im mimng a class stssion may ^ the %^ at some later time- After 
b^ing vtuvwt^il at fin; off campus locatitm, the tapes are returmd to *a catijHiS, er^ed, then reused to f ecord 

othet Lias> 

The SURGE stiuteiits are required to compfete the same assigitmnts, r^rts ami examiitaik^is as t»te 
Ofi-campus stmHmts. Uborafory work is frequently required in electrical engineering courses; the WBGB students 

the «aboraiorv f acihties of their employer to perform these studies- The students of many courses mmi 
computer f acthbes to complete assignments. Here ^ins^the inAistriaf computer fadltties are uiilM. To 
mmimt^» the inctmvemence of limited (ibrary facilities, the f^lty frequently send a sin^e Xero>s copy of 
reference artick>s to earfi of f campus secticm. 

In the 1972 73 ac^temic year, 34 industrial and ^vemment f«ilities participated in the SURGE fw^oyam. 
Tfiese are hsted ui TiHj^e lU^ 



TABtE tl 

Industrial Firms ami Abodes Participating in Surge Pro^m 
Ac^temk Year 1972-73 



COMPANY OR AO&ENCY 




Adotph Coors Brewery 


uotcien, uotoraciD 


*Balt Bf of hers Coroor^tinn 


Boulder, Colorado 


Bell Tejeph«ie Laboratories 


r^orvucir r* n ii^rln 
LrijIfV^fy V^OtUfoviO 


Canal Zom Socrety of Professional Engineers 


Balboa Heights, Canal Zone 


F. & 1. Steel Corporation 


PueUo, Colorado 


Cobe Laboratories 


LakewocKi, Coior^^ 


Ctdorado (Apartment of Health 


Dwiver. Colorado 


Colorado State Penitentiary 


Cation City, Colorado 


*0ow Chemical Company 


Golcten^ Colorado 


Eastman Kodak Company 


Windsor, Colorado 


First Nat tuna) Bank of Denver 


I^ver, Colorado 


H<;«i/fett-f^ckard Company 


&>l^ado brings, Colorado 


'Hewlett-Packard Company 


Loi^land, Colorado 


*Honeyv^n, Inc., Test Instrument Division 


Denver, Colorado 


*I.B.M. Corporati<Ni 


&Hiider, Cof<»-ado 


Lamar Community Coltege 


Lamar, Color^o 


Lowry Air Force Base 


Ctenver, Colorado 


Marathon Oil Company 


Littletc^, Colorado 


'Martin-Marietta Corporation 


Denver, Colorado 


Mesa College 


Grand Junction, Cplor^k> 


M & 1 Incorporated 


Fort Collins, Colorado 


Mountain States B^Hl Telephone 


Denver, Colorado 


'National Center for Atmospheric Research 


BcHiicter, Colorado 


Nelson, Haley, Patterson & Quirk Inc. 


Greeley, Colc^ado 


Northeaster Junior College 


Sterling, Colorado 


Stearns- Roi^r Company 


Ojnver, Colorado 


US. Air Force Academy 


CoIcrckIo Sfirtni^. Colorado 


U.S, Bureau of Reclamation 


Denver, Colorado 


U.1S Bureau of Rertamatlon 


Billing, Montana 



/ 
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TAMe It. IfidiiBtrM Firms ami ^ei^m Partid^iHl in SURGE Ftagimm (ContUI 



COMPANY Oft AOEWCY 



UOCAtfOM 



U.S. G0otogtt^i Survey 
U-S. GeokigiCiri Sunrey 
White SancU Mi$$ite B^n^i^ 

Woodward Gavtsrnor 
WVomtn^ Hi qhway Oef^rtrmnt 



Oi^ytmiite, Wyominy 
Wfiitt? Sands Missile Range« 

New JVf^xico 
Fort Collins, Colorado 
C^M^ertne, Wyominy 



■Ortgifvil f^jcaticms for inlti»tioo of progrsm, f «H IStift, 



Table (t lis a surmrnry of ^ mmitief of tmir^ and loc^kim; mui staetem ^lutiliinerttB, butTi on-campiii 
tmd oH'tampus, for the stx-year history ^ fircigram* The importafit feMures of this tabuliitfon ate ttm great 
diversity of th« of ferings and ttie ruinU^er of sroaM, dispsrs^ study groups. The SURGE av^ra^ of nsadiing 3-40 
$liittents/sectiOf|2 ^rtd 3J8 sections/cotir^ is m shsw^p contrast to ci«»isicdl ETV style. An a vet of 12.9 SURGE 
students C3 ^0^ is added to an on^campus enroitmatt aver^of 1S,6 students, fhus, faculty t^oductivtty 
IS almost doubted by the achiittcKT of the off-c^pt^ learr^rs. This added prcKkiction is obtdineil at a n^onable 
marginal cost ^ !^own in Appendix B. 



TABU til 



Coli^ado Staie Unhm^ly 
1^ 1973 



QUARTER 


NUM&ER 

OF 
COURSES 


LOCATIONS 


NUMBER 
OF 
STUDENTS 
ON- 
CAMPUS 


NUMeteR 

OF 

STUDENTS 

OFF- 
CAMPUS 


TOTAL/VR. 
OFF- 
CAMPUS 


Fall, 1967 


4 


7 


105 


1^ 




Winter, 1967 


9 


9 


132 


249 




Sprtng, 1968 


8 


9 


100 


206 


644 


FaH. 1968 


12 


13 


2B3 


341 




Winter, 1969 


15 


14 


306 


320 




Spring, 1969 


13 


15 


314 


288 


949 


Fa», 2969 


15 


14 


209 


336 




Winter, 1970 


14 


14 


262 


29rs 




S|M-ing, 1970 


14 


14 


162 


165 


796 


Fell, 1970 


17 


15 


232 


403 




Winter, 1971 


20 


19 


2^ 


316 




Spring, 1971 


18 


16 


235 


202 




Summer, 1971 


6 


6 


67 


51 


972 


FaH. 1971 


22 


23 


410 


351 




Winter, 1972 


24 


22 


353 


284 




Spring. 1972 


23 


20 


331 


253 




Summer. 1972 


7 


10 


79 


93 


976 



2a "Iwct.on" i» 9 Bfotip Of itiidonttwiroHMiin sfliven course at ono loeat.Wt. t»ch 5*)4 f.ui. f«e.v««i ortqtn4\ rt^e 
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Tabfe III, Dtfarado Stats UmirtimtY SURGE EnraHment Sttmraaty (Cont'if ) 

FkMJ, 19/2 32 

Wmm, 1973 30 

Spftn«i, ^973 31 

SummtH, 1973 17 



Stutlents Ml The SURGE f wtMFam mmlhii as graduate siurients of CSU awi are charawi the reflutar 
rusulant, part time tuition rate whtcti is currerrtly $25 per «|uarft;r hour . The particiMting industries arnJ government 
agencies prowitJe pJayback equipment, dassroom faciUties adnrunistratiwe smion of the progrwn. No other 
charya is nwife by the unitfcrsity. The SURGE Students receiwe academic credit on official CSU transcripts. No 
nofcitiun is imtte to distin^tsh ott c^tmpus and of f-rampus students m the records. 

While thf'rw is no |)rescritj«l pattern, each faculty member teaching on SURGE is encourajy.d to make at 
kiiit tvw» visits per quarter to eadi imlustriaJ location for direct cont^ with eadi of his students in a class. 
A<khtu>nal lm« interactifjn iMM^ve«n faculty and stutlents occurs in occasional tele|>htme calls wid nKKe rarely, by 
stuctent visits to the campus. During the first three years of SURGE, the CSU Hun#an Factors Research tatJoratory 
cumltKted an wlucational evaluation of the program (7), ThiJSe studies consistently indicat«d that the students in 
the remote classes were attaining lewis of achievement equal to that of the on campus students. Later surveys 
sfiowed off campus students' attitudes toward the videotape method of instruction were more favorable than to 
other rations available to them. The faculty «id on-camfHis students generally have favoralile attitudes « vwll, 

Atlvantat^ realized in the videotaping of upper division and graduate courses for engineers and scientists in 
SURGE are IB): 

I . Videotape allows complete freedom of scheduling of courses at each industrial location on a iwo-day 
delayed, regular secfuencc. 

2- Videotapes may be retained for those individuals who would otherwise miss a class because of ill- 
ness or travi.'l. 

3, Students tjoth off cjmpus as well as on campus may use the tapes to review lectures. 

4, Courses may lie tauj^it at locations beyond the bounds of a feasible live ITV system (there is no ITV 
system ojwratmq in Color atto at this time}. 

5. The capital cost of the wideotapmtj o^wratica is significantly less than a live TV system capable of 
proviiltnq the same opportunities. 

6. Faculty may reyifw classroom presentation for self-evaluation. 

Dunncj the first .ix years of the SURGE proqram, ov«r 50 engineers of participating companies have t>een 
awartliKl M.S. degrees crjmpleti'lv through the videotape prcqram. Ovt^ t6,(X)0 quarter hours of university credit 
have {>een eanietl by other professionals without leaving their {jiace of employment. Other benefits «;crut,' directly 
to the faculty who have (>roftte<l from the interaction with top practitioners in their area of interest. Much a', the 
viability of the programs stems directly from this stimulation. 

To complete this discussion of off-cam|>us graduate programs in engineering, let me return to ttie problem 
of selecting the most rost eff(x;rive delivery system. In a recent review of the program costs reported for the 
Stanford ITFS, University of California at Davis, and SURGE {irogram, Loomis and Brandt (9| fxjint out that the 
dominant cost in all three systems is a<lministNtiv«? "ovprht»ari.'* In fact, l)etweftn S20 Si^ fw TV classroom 



24 
28 
29 
16 



527 
750 
367 
96 



426 
426 
275 
150 



1.277 
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ki^tutt hour must be assigned for prc^ction and program mana^ment. This is the dcmiinam factor in toiaf 
telteifision system cmt of betvi^en $30 and $50 per TV cktssriKini lecture hciur tfi^ three systems, 
FuriheritKire, anofihet $4 tu $7 per TV das^oom tecttire hour Is retired to outfit the TV classroom resardless 
qf typ«^ of delivery mode. Thyjj^ ^oi^e of delivery hardiMre is not as big a ojst issue as one might assume at 
the outset and contrcri of suf^XHt staff costs requires relatii^y nKKe attention. The decision |»oci^ on a 
delivery system in a given setting tt^^t proceed tiy first ejcaminin^ the eduoatk^i issu^ and data, if any 
have^en €»ily opiriKHH m> far^ i^cernlr^ Hte audio talkfcwk. Audio taHchack can t» surpN^isln^v expen^ve, 
fterhaps $1 to $7 per TV dassrocmi lecture hmir. If 

th^' (iec^HKt is made to employ a system i^ithout talkbad^, the decision as 
to whether ta potnt-to-pofnt microwave, ITFS or ^deo tape delfvery 
sy^ems, depends primarily on relative cost. In gener^, v^dtely ottered ^ti^ 
class sections tend to favor a vid^ tapf operatfCMi; ^^aphicatly ccmcxmtratmd 
$mall dass^ tend to favor iTf^S broadcast; mtd conmitrated large ^^ions wet 
potentfaily larger distances ^an the ITFS syt^tem tend to favor point-to-poim 
microwave* (9) 

pn-Can^s UrHter^adhta^ f^'ogfams 

The use of television to reach additional, well-qudtfied students off-ompus is a natural and non -threatening 
i^licaiion. However, the aspirations of and live of^ions availatMe to fuH-time campus studettts. as well as prot^ 
terns with faculty attitude, h^ limited the a{:^i(^timis of video ledvioiogfy in engtne^ing education on^^^mpus. 
On my campus, many other Jisciplines make far more imaginative use of vtdcK) for instruction than the College of 
Emrineering. Two interesting pilot f^-c^ams, howewr, may be of mterest* The de^ioti<^ wilt be brief and 
anecdotal, not the "hard stuff" now in vogue in educational tedinotogy circles. The. ^ i^^ograms are personal 
favorites of mine, thou#), so I will risk tfte hubris usually resent for a dem's pet project. 

UtKfergradi^t^ study opportunities in enginwring during the summ^ session on my can^Kis have long teen 
a dis^ter area. We cancel more cot^sef tf^ we teadi fkiB to poor enrollments ^d the courses which are taught 
opiate at the same cost as a re^lar session but with a third the enrollment. This past summer session, however, 
we confidently announced we would offer ei^t required freshman and sophontore courses and we did teach all 
et^t at less than half our usual summer session unit cost. At the Higgestion of samral enter^ising faculty mem- 
bers, ali ei^r courses were uped in a SURGE classrooms the cour% w^ taut^t during either tt^ winter or 
spring qu^ter. Copies of all dass handouts, notes arrd ^ot>(em solutions wmt filed for use with the videotaped 
tectures during the sumrror sessfor;. A faculty member met eadi class for a two- to three-hour problem session 
each week. One of the professors experin^nted with a modified Keller plan (10) in which the student couki view 
the tape on bis own time schedule, follow a supplemental fM-inted lesson plan and have opmi mrc^s to an under 
gr^i^te tutor. An extensr^ evaluation of the fKogram pointed to nuimrous ar^s In need hf improven^nt, tnit 
CW1 the whole, the sti^Jetits understcxjd that the videotape was a rwe^^y Ingred^ient to a^ure the diversity of the 
offerings. They reacted much more favcK'^fely to this experiment than they did to some of our earlier, ill-fated 
attempts of several years acpo which occurred during the academic yc^r. 

This commg academic year one of the faulty merrrf>ers plans to offer a lower divisw* elective course in 
this modified Kflier method on "introducticHi to*Ex{;^imentdtion,'' packaged bry C^. Er:«est Kabtnowic^ of the 
M.IT. Center f^x Advanced Engin^ing Study. Later m the year, he will offer an upper division elective course 
tiased on video course materials we are rec^jesting frcnn the Sell Telephone Laboratories. Perhaps this enriching 
mode of intrcKiuction will errcour^ the use of quality video packages leased from other so they will find an 
apprc^riate niche in the regular camjxis offermgs. 

My other pet fwoject is a freshman engineering ptogram which has proven very popular mvr a six-year span 
of develc^ment 0 1 1 They key factcK is a strcmg personal asocial ion which the stiKlents develop with their 
engineering professor-advisor during the year. A spMific group of 7530 freshmen are in$truct«l for the entire 
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v*Mi .Jf> ♦fUtHjtated ihrtH- ipMiter a»*tftt onnt;** hy rtu; s.4mt' tnufinsfonny proft»s*»ot. who atsr» serves ;a their academic 
jtlviMU Tho ^iti%p:c\ nuifttn ijiuitN im** iiom^mwr \noi\ramnmiU tjcuphics, f<?|Kif I writing and fifojuct design. That's 
r^fjHt t?**vtfu:.<l imi|intH>nfiq Xt\wU (|i,H>fi^ as m^rsiHKiJ M<:r=fH:<! to miconra<j*j thi' stiMtent faulty fflatiiJii' 

sfif|i. Th«* *!vi't.ft|«f nimilK*r of uffn:i' lonsulf jtit^is wliK:h Htc ntuiiiriits nudu hist fall quartet wa© 3.4 Ffguif* t dtitnon- 
!*tt.in«i> tfif %Ti niiqty f,ivntal>U' n'v|H>nv ivliidi tins f)iofi!^SiH ttilvi&uf ^iditjmi? fm?ivc^ on art mci^pemlontiv run. 
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Figure 1. SattsfiH tiofi st/tirtyy r^fsmHs for fresivmn cngmmrnrg course. 



The fHltirjtiondl l«*rhnoln(|Y <:uri|K)n*MJt of this proqtiim is *in nutf^rowth of inci eased engineering faculty 
dwar^'fiuss of the pfc^)U?ms inciirftvf hy freshmtiii. Folfowinq ^•riotis c/imfilauifs fha! the ri?si<l*?na* fratU were noisy 
(cr.g,, '*My rtKimmate VN*ho is stiKlying X n^^v* r sfudtf^s/'), wt? h,id onv. w<m| (of four) of a co cducationai dwnniftofv 
ri»ntf)lox cfosifst to the Enqinwrinq BiiH^fHtq nrseivrd fof imyiniM*ruH| d|)f>lfcafits, A lounc^ area was outhttetl with 
tUackboarcte, clectffKiic CilcUtalof^, te.i^rma^ !>ouks, and mann**d i^ycifarly hy uf uiertf a^fuate tutors cfiosen on the 
Jhjsis of hn^i acadwnic abUity and qood f>i*f;onal5!y . AnothiT lonm hous<-d a C(impiin»r Manaqed teaming System 
tt«fm«fiai <12) which offered instructKHt on tht* slide rule afuf a coni|Hitt»f cuffJ piind). 

Each room in the re^idpn<:«? haM ha« an outlet to two CATV sy^tiw^, t>n*' cd fftv au'd mnunvrrtal awl tTV 
t3roadca<;t statiofis and the other channel Ofietaterl by the Coffi'i^c* of Entjineef inff as an f TV poDfjram to su^iptement 
classroom vjudn offennqs. Three jtfLinnel*; wfife ret^tarly otieratefj on ^wvk off<!rinr|^, Three cftannels were regutarly 
operated on w*H?k tiays from ai^^ut 10 a.m. in Tdn-m la*;! academic year Onv chansuH r ontmuousfy presentetf the 
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day's schadule and s^iedat annoumxmem s com^rning cam|^ events, ^minnrs, siuctent society fneetin^is^ The 
otiWTHwo chaiiraH repettisvefy pr^s^nted caordtnaied f:Hr0grams on ogAs^b a^M^B^ cdU:uius, study skills diid 
f ORTP AN, To incre^ vKswtn^. the graduate ^udent in rr^eorcMogv presented a focal vi^attier report ihiw tinwi 
a week. Fifteen en^neering luienlatian films obtaiited from various technicat societies^ ^vemmerri a^g^cies and 
cellar untverstties showr> re^M^^y dirm^jhout tl^ ye«r. At the of frit carter, 97% agreed (67% strctfigiy ) 
that ^a Si»!Ciai dorm for en^n^tng i^tuitents is a good idea/' A total of ^% sigrmi that good use vm msKie of 
the special facihtt^ provided, frankly, t viras ^rr^hat dis^tpomted dtat the si^ementaf CATV Instnfctiof^ dM 
not receive better use, CATV imtruction faired iKioriy compared to tfw tutors, tftoi#i t suspect CATV dwl quite 
VveK compared to the uniirersity titx'ary! But to our surprise, the students nwie ex^fent of the engine^ing 
orientation filim without the sllght^t faculty em^Hiras^t^^. Ch«r 68% of tt^ ««udmts in the, itortn wat<*ied tite 
orienlatimi films (whk^ are a mtxed tot!) vohintariiy, on tf^ir ovm tfm^ and reported that the films het|^ them. 
Pleiise tear in mind that tr^itionatty tl« most unpopular event in ihe life of an er^neering fr^hrMo is to be 
required to watch these same f ifn^ in targe lecture haHs m v^t traditionally was called "CNsan's Lectures." f 
lielfitve this hint is myrth systematic examitKitiork Career quidiNice ioAtmmtion mi^t be welcomed cm community 
md campus CATV systems. The surveys I have sem convince me that it would be very easy to improve on the 
traditional faculty advisor in this arena. ^ 

Scmie ChaHen^ng ExtenStom 

The SU RGE program, ar^l oth^ like it, dc^ not s^va tt^ edt^tional needs of en^neers where the 
TV signal reaches. The f^ogram a^um^ that the off campus students can attend and team in the usual campus 
graduate course. This is c' arly not true« The matfimnatical badcground in use by toi&iy's studem far surpa^esi 
the original training of most 40iikis year old en^r^rers. This point was brcaig^t home to me in a recent survay 
which we conducted in one of Coioracto's lar^st aerosf^K^ firms. From a profi^ional staff mailing list of over 
3,{KK), we mailed a questionnaire to three grtnips: 

Group A - Alt ci^mnt SURGE students. 

Total maii^: 65. Returns: 45, or 69.4% 

Group 8 - A random sampling of 70 former SURGE students, men who had enrolled 
before but were not longer in the pro^^. 
Returns: 20, plus 9 umtelivered, or 28.6% usable returns. 

Group C - A fanctom smpling of 2S0 names from the entire fist of over 3,000. 
Returns: 76, 30A%. 

Tdble IV summar i2es the responses tiy age, marital status, avid de^^e^, ^klte the striking skew of GrouF» B and C 
to oi(^ employees in what is otherwi^ a similar tot^ ^mjp. There is a r^ed for specially designed courseware 
for the older engineers which the gr^uate curriculum does not satisfy- The jM-oWem here is l^gely an economic 
one, because the univa^sities do not have on-going cam(H^ instruction to acW these potemial students to. If you 
add the usu^ direa cost of instruction, say S1(W lecti^e hour, to the victeo system costs of SSKJ^^O TV lecture 
hour, ihc funding problem ^ts tough. Yet, time employe do need an opportunity to revitalize their education 
at mid-career, because the social costs of ftot doing so are great indeed. Perhaps the federal guvemment should 
examine this prc^lem and identify an ap|XO|:^iate role for its involvcrrHmt. The problem is of a ''software cost" 
nature, not delivery hardware or methodology. 

A second opportunity is in the area of career guidance. The orientation films now available are generaUy 
margif^al at best. We need*honest, tinwiy presentations rwjt the heroic i^opaganda of yesteryear. Tfie nij^^jriunity 
to inform young people about the career options open to tt^m throu^i the use of gtii<l»nce films f>n CATV is a 
real one. It is information that young people seek which can supplied via a rt^ium which fits their lift? 
style. Quality films are nemled. 
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TABLE IV i 

Survey of Engttteers at a Utikijfit Oiioiado Aerospace Fimi 



f^RC£NT RESi»DNSE 





GROUP A 




20 2}jf 


20 


10 


20 30 


38 




3» 35 


27 


1% 




9 


15 


41 45 


2 


2S 


4G50 


0 


• 


51 &b 


2 


0 


56-60 


0 


0 


61 -till 


0 


0 




2 


5 


MARiTAt. STATUS 








89 


85 




g 


10 


No Resfnmsi' 


2 


5 


OEGRCeS 







AS or AS 
BS 
MS 
8A 
MA 

Othf-r 

No R«*^}Kins«' 



0 

2 
78 
16 

0 

2 
0 
2 
2 




G5pOPC 

3 
20 
22 
13 
20 
16 

4 

3 

0 

0 



80 
16 
4 



3 
3 
57 
18 
4 
1 
4 
4 

a 



Thf o\wn le.irfunij conct-pr is rccetvinq a great tJeal of attention now. ft»e structures are beginning to take 
shsfW! in tjdrts of the Unttod Slatw. I hdv«; vet td hudr of an open teaming sj^stem which woufd employ the open 
door ur TV stiKJio-cldSsroom t<H:hn4{juH alreddy »»o¥«ii economical and eti^cationally sound in eoginef^r ing graduate 
fK(«irains. Why not ad<< stiKtents to Mminjs dasses as we do in SURGE /nd reach these people in their hemes via 
CATV or m stiifly <irouf>s at their place of employmimt? I have heard much about the shortcomings of "the face of 
the t(ib»\" but I have yet to 5e<! a plan which would increase hii;^iBr etlucational productivity in a more natural 
fashion. Tlie ofivioiis advantaqt- of the SURGE methoiki'ogy is its low cost arwf ^reat diversity of courseware. Many 
of our U.S. universities have the traditions and al)ility to explore this simple form of extending classroom learning 
situations to nearby locatmns Shouldn't it at least Iw tnwl as a pjM of our g^jmrtal expltjiatioo of tfw oi>fin learning 
conc'f )f ? 

The public poln:y and adnjinistrative issti»?s associated with tht- intrmflJClion of educational technology are 
C'imfilex. But as a mid<llt» level inanagin m higher education, I ant nnore conretnwf with old budget practices which 
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i^eciude a fair hearinq of C60tni€dilv based instructkM^f options. The total lack of (^nskfe^^dlton which is givtni 
to Gd|}itat outlay wiwti^atkuni m iHidget preparaticm ean nc (tHi^r be ioteni^ In a Rowing system^ physkal 
piant cOfTstru€ttofi niay tic fmanced ovc^r scwdral yeam by boi*i^. But rarely do pyblie officials make a decision 
from altemattves wHii^ recog^^e that mcmt t^nicaliy tmed (^i^n^ are i^itai mtemiue and wMt not survive 
a budget eKercise v^ich focm^ w minimum experttJiture £fraf yBm". State officials strive iiu^^easin^y to contrcrf 
annual expemltturcs and ri^tfully so. but "tl^t budgets" are wedded to fix^ labw field f^petuate the 
^pirding costs of a fafaor intt?nsiv# system- 1 believe that off tctals mana^ng fmteral pro-ams must recoj^ke that . 
mndvational diffusion (s ^neatly inhibited by our inability in education to frian capital expenditures properly. 
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SURGE FACiUTIES 

SttufHf CfjssrfKt^m (Vm t.t tfu- 1, ,u. CSU sti«f io ci.(«r«u.n., is ,llusirjte«l .t. f^iyuri' 2. Eucfi classroom is 
iH^m^^ii w.th .,t kM^t tlu,.,- iumm^s. A t amura iwr lti« imtiottot'sciesk dJfows htm to tlisjiJiiy wntten or 
»<Ui5.t.at.«.. »iiuit..M.,1^. Tf... .fwtfortu. aintfols «„• owiilnwi caifltna fm ftinct,om sudi as /oomiiifl, tocirti»T|| an J- 
a.ni|H,s,m,. A mofi.tm at th,,. mstru<:tor's dt«,k disi»lav* ihv pH^u..' getit^t«f by th(, owHreod cam«ri». 
A,mt»H.» aiuH^r.! ,it tf^) iM*:k «f t»i.. nnnn ,8 wuumtnl on a (mi t.it ht?.«l, this cameta is cuntruliuU Uy « t«chnician 
.« tf»' n nttdl irt f.,.:,),tv. Tl«. fcchnicijn nwy ifmot.'ly |wn t.U /oiiin uml foctK tti« rtjat camera. 0 th.. 
msltia-t.H tu t»u. ft,„... (Mn,^ chalklMWfcf o, walks ;if(H,mf tfitr room, th« techtiiciJ^ follows h« movemwus 
A W.I,- .««,(,., t,x.ul .•. nu.fa at th,. frtint of tht» room .s li,rat»Kl Co rm:k up a sfiqm*mt of th.> stutteit class. By 
m,MnN of svw.t. h,". .,f M,.. .r.structiir's <l.'sk, th« uMnictn, sel.'Cts ffie caint^a wfiioh is to Iw recorrfml. Two rooms 
turn- ^,lii-.cK-,.i. .:.»,MtMl.fy vvfiu^l, allows ftu- tmumtm to sho* two fiictures on a Sm.jle scriK-n. Two monitors 
au' l.«uif,ni at tt». ff ,«,t <»f tin- mum u, Mkm, <itu«leiit!. m ihr class to s,*hj all rmtetial fwesentcri via the overhead 
c.inu«a. A iHiSfon is lox:^u%i m front uf imcH ^ttKJont to activate an ov.frh«ad mK:ru|)honR, which ,>tcks the 
mH'sUi,m ,,nd othijr dialo^i*- lwtw,vfi thtt |>rnfnssor a.ul stmieM^. B^'catm mniv,m soowtimes fad to jkisli th.? 
mtamthiHu- UuUnn, a sr.nilai huttcwi .s locatwl al thrt icwtrortor's diiik amj the t:i»nsok. at the recordinq h«ad- 
«H«arf«-jS. 

Two of thv sttulio c\M'^vom% ..r« tdwitical to That sfnw/n in Figure 2. Tlwstf rooms each s«at 30 studems. A 
third .t.id<v.-da«rmHM was tu.lor^l for thf ^M-ds of thf hujhly inter iict.vt' iiKtructional mt'thods whicfi are common 
HI the Collwiwof Busuurts. This s^m.nat tmmi a<:comfm«laf«'s 16 students around a larrju oval faille. The TV 
«-..ft,..,j a.raMff.,nent .s s.ni.laf to that <Hitltn,Kf ..Ihjv... A fiHHth <.t.id.o<lai.sro„,„ scats 125 studtHits in a small. 
w<Hl(i«' slUfHHi auditununv 

Control Con.<iaf4^ £,„ , t.Ltssf cKJiti has Jfi tfuhvidual nofisol.! at ttw r<jcort{ing facilitie* where a techmcian 
(iu'^iwnUy wMkstudy u.id. tfji acfuat*,- sttKhmtJ i% ..,rH>lovHl whenevcf an msftoctor <s in the classroom. Figure 
3 vn-w of ttifi area. Th.- fochnician .nx'raftf!, tfu- luick camt-ra f Ivctron.cifly from tfus location. His has tele- 
>>li.Mi.; commo.HCdt.ons to thf; piofrssut jfirj o«..T(id« fhf audio qa.n. Courses an> not r<»f ears«f . thus, the 
tiH:f.fi.r,a/. I.s!.-mn fc, fh.^ rl.iv. MfiOntdtfon add t.ik*...; v.m t.al .:up<, «) as to disfilay thf fK*st jMcture pc^sdiU-'on 
vt(k*«i tafJir. 

Rfii^r./ Arm Tfu; f..-cord fac;i!t?y shuwf. in f ,.jurt> 4 nonsists of thirty-two video tap' recorders and 
moii.ton, uu wh.rh f .1,.,.. .,r,. nwdi- for thf rofnott' locations An onoinal vnU'O tape record is mad« as the class is 
WHMluct.'d on camMus for f.ch off raminis vctiofi A sw.trhei (h-sHjnwl by thi- telovisifm staff a used to ,>rt>gram 
tlu! nmli-d ifundJ.^ of ,(Hz(,ui>',% Um .,ny c,,v«'fT cou»sc. Tfif? numl».:r of tafws m-ediMl for each dassftwm variirs 
ff tirn <uM)t fi) iiuiij, 

r,we Mmvy Each v.dixf tain- in tit,. mv..„t(Hy li «iiv,..i a numh.n . A card catalotju*- is mamtatned on all 
fa,,..; MTUfdin., th,- foctii.n <.f .-ach virleo tupf within tlui sysfm. TafH^s are (wckaged in fifn-r shipping cases ami 
addnssfd for fh« ^nam di!i.{.it.ition f arh o«»'ninq the ta^MJs ar»^ picki-d up hy comnTfirca) courier (Unded ParcfH 
S'lv./-- 5 f(,r d.'livety the rn-xt day .iJ r.H;h wmm- kicatimi. A|.f«ox*matfly 400 tafjrs ar*. sh.M{H?d tn thi? »,-mole 
Ifjcattons vvc«'k!y. 

in Plant Clussroom At mou in plant locattortN, a muff t-piit(»os«' ronfori'nci' room wivis as the feqularly 
<i.:hodul,..d SURGE clas-iHkmv Ont' or two ?3..n<;h TV nwi.tufwnd a v«l..orw;or<fr'r arn tlw only special eqiiipmi^m. 
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BEST C€r/ .n";;:iL/:::r 



BEST CiPy lUMCzil 
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S£ST COPY Alf/;iLAi:i£ 



In 1967^ ^ynfteVcK^fH::)^ etnpkiv^ sptely, biit less Uim Mf of the tc^iKms ncwv rety on 

this filacHitti^« The mmt pbpiiidr m«K:him in lise today is th« half-inch Ei#S marine/ t^t it is Iwtr^ faviM' to the 
thietef^quaftte'indi C^s^et 

Tht imtttutional settir^ Can 9^tly aflMt 1^ cc^ of lyny program, hi orcter to prc^wiy qiiMify ^ infor- 
niation tof be presented ocmoerrif cos cwi a si n ing the Col^^^ 

Slat^ Unlv«rs$ty4l«^sng is 

CoW^bStateU^^ 1967 dM not fw^ a histil^ 

ctmrtH ^ff^^ettmit^mi^pm fc^ part-time stiicferitsi. Con^i^tfy, the univ^sitylHKt^^ 
to fnana^ ^en^l extension. Faculty w^e ndt a^^Lmomed to extra pay for off-^^fHis or evening iimruction.. 
When the SUjRGE fKOgram ifl^ intiatedby the Ck^il^ of Er^n^i^^ rnte^^te 
the activity into.the restctent ^n^h^e {3r^i^^ of the ctepartments. T?^ {^an of tlie {graduate Sdvixit obtain^ 
i^p-oval f r€Mn the fiKAifty <^mmit^ iscuit/ ^ymmmi for i^ ct'anges of tracKtiomrt ru^ nei^ssary to 
^rcomjplish this irttey^lfior^; The fa^ty, tt»^ jK^-tirpe «tud8*«ts to r^futor ct«^ tfvith 1h^ futt 

exp^c^t^on^lhattr^litionitf ^&t\6s^ ijf gf^^u Sf intent attainment wouU be met. It wis at» p6^bte to cbuAn 
the full fmpact of increased faoifty |MrodiKi:1ifity witfumt the Imrcten of a fwi^ aclnrtintstrattve stri^ure. 

Oh thfe other hatid^ the l^5foot0eni2emt^^iaiupf»n^thintteO^^ 
IgdiK^tf ona) Media as a central serv^ otganii^atfcm. Tt^ furxrtHOn of tfiis or^izdfion tt^to {Moviicte Untyersify* 
wide iiiedfd support throi#i tte 

Teievi^on Services. The total (evei of expendtliKe of the Offieeof Educatioi^l fttedia for staff and expenses iit 
Y972^73 wes appriMimateiy $5QO«0IKI. 



\Wien tetevisiw imptenwnted on camper at Colotmfo State Uniya^sity in a tdevtsion poKcy was 
ikiopted by the governing bomd v\^ich ptac^ ttn mifXHtsMUty withm the Office of EdiK^tional Media for: 
9^nersi} sufwvisxin of ^1 TV pro^ems^ pio^e^e. invc^pry em) f^iiriii^in^t^ of eft eqM^»nent« of^itjnif^ 
funds for the fmidiK^icm and distribution of televteed nr^feriah for r^deitf mstn^km. tVofei!^^ staff vuere 
anptoyed fw the tetevisii^ operation. tAomQy&, staff of tf^ Off tee of EdtK^tlonai Media had ctenxmstrated a 
%Uof^ vviUin^im and ^Uty to deihm. Thus, the ptac^mni of this respomil»iity not a function of 
antthority btit railt^ 3 nmns of tribtainiiq a aH>rdmBi^^ weit^nami^ proipimi with an ih/c^nce of sfriimerins 
and dupfieation. 

f^olects soc^ m ^RG£, CQ-Ti£, Ht-TtE and ot\hsH haw imn smoothiy int^rated into an ^tve can^iM 
pro-am. TY^e are seversi adi^nt^^ to th^ f^ocedure. Mii^ ti^ ai^ratus ami €thm effui^narrt mteded for the 
on^mp^s program is avaitebfe to used for the new off-campus tn-o^am. On« hi#^^y o^itfied leteyision m^rmBr 
is abte to d^i^ the off -campt^ fmipisnfi acnnponatts as vi^fi as the powf ng campus system of TV. Video ti^» 
r^ord^ ami reiated iten^ are bid in Unh^ersfty-inmto <{immft^« tht^ r^uc^ cost. Technical standards av^ ectu^ 
ment i^mpalibfiity is n^intained arMi tmpro^ MtUi^aticm of both ladtities and staff is realijsed. As the off^c^fi^nis 
fx'ogrann at Co}ordCk> State Unl^rsity expand ktto ms^ d^pUms, ^ indusidn of S4^ effcH^ under a 
servk^ Ofg^zation OH^Inues to be ^JvEntageoiB. Of comm, ait academic dectskrm remain the re^Kmtibiitty of 
ttie academic d^itmdnts and ^tli^^ irnM^ved. 

cost of ir^ri^tiim for the S^RGE fn-c^am whid) ^ves f^Kticif^^gtf^m ax^t i»'ole^onals in 
industry can foe estin^^d v^h precision, l^<^use the pro-am has compfetid six years of of^tton and attaiMd 

^TUrt rnat9fal ^pi^f#d in tvwo previous unpubii«h0d CSV fepott% (rftH. %^ and t4^, / 
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d ref^fveiv sta*te level of actMty. Ttie toUc^A^kig dii»^iHi coiM^itoraies this in^c^m ^ttwi#i mudi <rf tHe 
kiformatiim on the cost of operatkHi Is itfifMtefifete ipmr^ty to ^ re^d^iy sdmfeited courses ykleo tefied fw 
Use off -cami^ 

Fdculfv f^rockrt^ivitv m^sured interrm of &tiHtent-w^t tKKfrs. iiKireased siipiificantfy in SURGE courses. 
The five-year avera^ enroiiment of sti^toiB per cotir^ is 16-2 oivcampus ami 18*2 crff-rampus. Tlwe im bwen 
no ad^it^m^t in f ^ity teacMng sdiedules due to this ^iticmal k>dd« Rather. grmUtig ^ yadi^ i^i$tai« 
help h^ been mppiied to accwHtu^is^ the Uicreas^l miroNment. 

The "dir«i a>st of tn^itictton" orhc^i^ms in ^ traditiomil mode a defined hw« as tiw hmnictkm i«t 
(fectmy salaries) tlivided by the total mmi^ of stiktent ts^mm oredit hmm assocmied with tfMit k^tnidtoa 
This index t$ fr^ii^mly cited as a fma^i^ i^ftrt far amif»tf^ re^i^urt wiatlom betMMn sMte^ ^MKris 
b^iMen ^oii» discfplirws* C^rly, atthough a dominwt cost in most imtniction, this fact w <to« mw ft^pmsm 
^ total cost of imtructton. The d»ect txmi of instrm^toti cmHan^ in ^ CSki Colt^ of Engkttali^ 
d¥era^ «wer aM levels of instruction in 1972-73 h $37.46 per qusrter stodent oedit hoyr tot, er,!- Dimpar^ 
data gathered by Dr. F. E. Terman and average over seventy-of^ engineertng colteges yields $49/qt. a. 
U £r^, f c/., pp, 510-S14, (1^1; a six p^ cent anmiai incare^ tras assannxi to update thwe da^K It is 
wid^y reco^j/ed that gr^uate tewl instr^titHi « ^^^orei^ty more ccRtiy than umkr^mttai^ dwt wkte 
varietions in this iralex are usually found tietween ^mes ^Mnal sciences. F«r ^ M.S. tevel 

SURG€ courses which are |:9edomin^tly in ^^neertr^ «id mathwiatita, tfw CSJ direct cost of ^struction 
<»vcampu5 ha t^ai ^timst^ to tie $^7^. a-. This figure is vievi^ as a cor^erv^ive eairmte for gradi^te 
instruction which av^ges 16 sttKtonts per ctHirse, Any effective, mm tridittorat imtriKtk)!^ sy^^ wouW 
generalty to expected to cdmpete with 9B6Jq!t. cr, if introduced on-can^nis. We af^ a con^ison 
directly to the off-campus tnstruction of the SURGE program. 

The cost of the SURGE proyam cm be divided into ttwee ixroad Gateg<mes: 0 1 amonlzatiw of equipment, 
recording spa<i and tape. {2} operating cost of ^oduction, delivw^ and i^opim administratkm, ^ (3) 
incremental direct instructional cost of addli^ off -campus stiKfents to existir^ classes. We d^cu» ttwse cost 
categcx^ies in the foHowif^ para^^aphs in a nwiner v^idi makes the ^Itng laws of the program c^r. That is, 
we will focus on the cost of recordii^ a class hour in a studio cl^oma pim tf^ cost of making and cWwering 
tape copies with instructiohai support* 

i:qui>ment^ fteco^inp SJaace Tape - The cwt p©r hcHK of recordir^t dominr.ted t>y tte an«>rti2atkm 
of the $25,000 for tjhe ren^odeling afxl equiwii*^ of a Studio diwocxti and control condyle. Tabte B-1 gives the 
ctetaits. Assuming five-year^ anKifttxation wth six per cent interest aixi 10W Niurs per y^r of utMiMtion, we 
calcufate S67hr. I^tote thai oi^y the cost of tt^ TV facility is consiifered, because a regular on-campus cta» must 
held in a d^oom, and we are interested only in the direct cost of adding off -camfxis studmm yia video taj^. 

A video rerorda- and moniic^ iSSOO) was amortized owr three years assuming ItMJO hoia^s/ye» of use and 
six per cent interest. The resuttant rost is $0,3D^r. 

An hour reel of 1 /2-inch video t^ purch»ed in large lots costs $20. An average life of 100 uses yieWs 
$0.aB/hr. The video tape ca-igi'^l copy which is costed hwe is mt distributed to off-campus studmiis, but rather 
serves as a redundant or spare copy to insure system reliaoility. 

The central recording fwrility sp«e whidi houses the Mntrol console and a sin^e recording unit is aX) sq. 
ft. This space was valued at $30/^. ft. and amortized over 40 years with inten?st. This cost increment is SSJ^r. 

The total cost in this category fw recording an hours' class tinw is tfie sum of the ateJve or $7.10/hr. 

Each ^rtitiona! tape copy requires ^.M/iape for recorders and nwnitors and ^.20Aaf» for tfie tape 
inventory. That is. the inCTemental cost in this cate^ry for each additional tape is $0.50/taix?. 
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TABLE B1 



t Ran tilt ccmiirai tinit 1,100 

$ TV motiitors 800 

2 Zoom lenses #TJt» ^ 

fmtriictin0 desk wUh cwtrot unit^ sfrfirs^^ 

^heratoi^^ and t^k 4J(X)Q 

{i/teter Controt p r - j niUh TV n^nitars^ 

swttclifng unit &,fiCK) . 

necessary reimdelmg ./ S^ffXI 

TotaiCost $2Si(KX} 



Qpmitk^ Cost 0f Office af Sdaosiiofm/ ¥edm - The imit opiating hiK^t fm {rfoduct ton ami f^^o^m 
nr^na^ment at CSU for tfie current tevet of ^tivity (80 cM^t&^m) i% S^«3D0. ThlibtKfgsf ifK:fifdtes fr«lk»nat 
time of an administralar^ a f^o^^ ccM>fdinator and a TV en^neer phis two fiilt tinte TV f edvi^tis and a 
s^retary. Other direct costs inducte stiKtent laixK. ^i^^pttes and spam parts, trai^ mni tetefrttc^, ami pros^am 
corre^Mindemse. At the levrt of 80 cowses year or 2^K) retarding hcnrs, this base operating tMfd^ m $^.26/ 
recording hour. Detaifs are given in Tabte B-2. 

A recent survey of both the CSU vt<-eotape system^ the ITFS hroadcast system of Stanford University and the 
microwave link of tf^ Unimsity of Calif otnia at D&m and the A«E.C. facMity at LhfC^more Stowed these ''overhe^ 
to be dominant m att systems, (ref . fdH. 

Each tsfm oopy of an hour's ler^h requires an ^lditk>ral S0.SO fcN" tape hiPKMn^ and $2.50 for ro^rKl-trip 
delivery by c<mimercfat courier. The inaementai cost in this cate^^ry for e^ delh^red tape is S3.CX)/tape. 

. Qperatifv Cast of IrmriKtmn - The faculty does no! revive any addittonai ^y or work aNowince for 
teaditng a re^lar campus ctas^ m the stiftlfo^das^ocmi. So there is no instruction^ c<^t for the recording how. 
Rattier it is the instructional supfKKt of the off':.ampus students whirfi mxm be estimated fwre. The marginal cost 
of addint 15 students to a 3 quarter credit a>ui $e is assiinmi to 10 hours/we^ of graduate tewNi^ a^istarnre to 

tf» i»'ofessor with ail a^^s of the instruction. This aUowar^ anwunft to Si .(9 for t^xii off^mfHs student 
who views an hour te^tgth t^, or tf we let S be the aver^ of f -camiats emcrfknent in a SURGE location, SI .00 
S/ti^. To this im^err^ntal dir»t cost we add an ^fowawe for secretariat si^sport. ^ippltes arKf telepfH^ne of $9.30 
for each off -campus sii^ent v^io views a tape, or S0.30 S/tape. A travel allowance of the direct a>st of faoilty visits 
to the SURGE Icxrations is Si .^/t^e, tndependem of the r%imber of students in a location but directionally 
P'oportronal to the nun^^ of locations for tape c^tes made ^cb recordir»g session). 
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$ f» the amafli^ stoctent fra^^^^ 

Tot at Co$t of S^RGE Insttnainn ^ Tte tmtmt outHned^ive era swumrised in the firttovy^ ttM^' ^ 



2,40a 



kfiVSL 



Administrator 


$24.ooa 


T/IOttme 
1/10 t«ne 


i 2.40ft. 


$ 2A00. 


Cooitiinator, 


$i6;ooo. 


%tfme 




124»0. 




«ts,ooo. 


l/Stfnw 




3.0CO.- 


TV Techniciam 




3 full time 


21.000. 


32.200. 


$8cretary. 




1 full time 
fun time 




8.00O. 




dOOQtm. 
8000 hrs. 


6,000. 


i2.Qoa 


JrmreA and Tetephofw 






3.000. 


3.ooa 


Suppitas and Parts 








11,700. 


Printffig and Mailing Annouiw^rmtti 




34X10. 


3,800. 



Attc^toJ to eadi nmtBT tecmdinq m\ htmiy basis, ttmss OMt fiwfcn am: 
60.300 



or 



4,800 



Eiqsdmtei <^38ratfon erf 160 eowses annu^y. 
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AVERAGE NUMBER OF STUDENTS PER SECTION PER COURSE tS) 

Fim^ B-l. GS(/ Off-CatrnHts TVCo& AnaiY^ 
1£n 



BEST copif mmit 



SNCCOlRIIIMtt 



DOLLAf^ Pen 

C«LIV6RfiO 



EffuifKnent, Space 
and T«pe 



7.10 



cmiee of Ed ftitedia 

C^atint expenses 



3^ 



Instrm^foriat 

Operating E)q3enses 



1.^+1.30$ 



Totjt 




tA ISO eounm/y/ ^r. 

The unit costs of off-canqp^s immffition for tiie 1871-72 SURGE pTOgnam can be eomfwrnl from these 
costs factoff. The foltoMin^efio^knem ami 

Total Cour^ = 6Q 

Total Secttons* 

Total off-campus enroUment => 883 



Tteref en's. N = 



261 



3^ ^ecti««l ^ '/^_\ 
\aHiree/ ^recording hour f 



883 



S « rrr =^ 3.40 



261 \iectjon / 

^ let C ^ 1.(X)qt. eredKs fi^anted for lOcmtacthoursof cuin»«voitc. 



( 




) 



where if . \ b si^ivi(ted into ft)»d cost/cowse how- piis veriabts oosts/co&r» how. 



1^ reomiing hiHir^ 

(IOC) • /$32.2S [$4.7S ' $1.30! n\ 
V N • S • C ' 



2.51. #47.5 




) 



If 13. 



322 S 47 5 

+ + 13. 



(3.78) (3,401 3.40 



F = $5t.97/qt, cr. * $S2./qt. cr. 
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Ri^CiiH t\ui thr on campus fn$truct«on in these courses Kas a direct imtn^iimul cost of S^/(|l. a. m 
jvi'itif^' cmii Sf* t»fu*ilfimmt hi 1971 73 of 15.6 stu^nts. The avera^ cost of instructHin for cm-cdmpus ^1 
off i:jm|His imtntction ts: 



^ 156 4 12.9,52.) 

' IS.e '12.9 ^ S59/qi.cr. 



Clearly, the pra^iratn lowers the instructional cost from the point of view of ttte universitv and al^i increases 
faculty f»^cKliK:t(vity. Ekit it should be stress that this is a very narrow view of Ifiecost of tfiis ina^uction. If 
the ijart'time student had commute<l to caii^us during his re^iuAm \wrk hours, ^ dmxirtunity cost which is 
neglecteil here would flaw? \mm large ernleed - perh^, $300./qt. cr.! Fiwttiermo'B, the university nnay have 
been required to add camfnis classroom Sfw» and parking. Even more import (Kit. the cumulative cost of tedmical 
obsolescence to the national economy shoukl be estimated becau^ advance engineering mtd manaf^ant train* 
in<f ffi*quently does not occui unless tjro^ams like SURGE are initiated. 

A convitnient graphical f^esentatfon of the SURGE prc^^ cmt armtysts is fpven F^ure BH • The nr^r^nal 
oist per quarter cretlit hour produrad ^F) is ^own ^ a funcii<wi of th6 av^age ncmtt^r of stuctents per sectitMi it) 
with thi' total nurr^r of s^icKis per course (N) shcH^ as a parameter. In the CSU exan^tfe detailed ^love, 
S - 3.40 students/section and N - 3.78 sections/course whidi yields F S^Aii^rter credit, It is mKth noting that 
video based prcHjr^*Tts for pr^icing engin^rs such as SURGE are far retnowd from the dasslcal ETV model. That 
IS. rather than SSO,(KX}/hour of (Koduci ion costs sfKe^ over milMcms of viewers, the televised et^neering programs 
must operate at near /ero fM'oduction cost because the potential audience is small. In fact, the TM5ER system of 
SMU in Dallas v^ill televise a course for a total of 4 stiKlents, the k>wa State University vtdrot^pe system sets a 
total enrotimetic minimum of 5. and the CSU record in this regsitl shows instarms of taping fc^ 2-3 total off- 
campus students. 

The Cost factors tabulated above are realistic and may be useful more generally thw sin^y estimating the 
cost of a specific fwogram in Coloracfo. For example, the decision whether to ftistall a vfdeota|»' system an 
ITFS broadcast system, is a trade-off study tetween subtracting $3,50 N {the cost of tr^king and delivering 
tapes) and addinq the amortized cost of the ITFS hardware on a Cdui^e hour basis to tfi« base value of $7, TO, For 
example, see "Technical and Economic Factors ih University ITV Systems/' by C. A. MartinVe^. A. J- 
Morns, J, Rosenberg}, and S. E. TaMmadg^ f^oceedings of the I EE, Vol. 59, No. 6, pp- 946-953, (Jurw 1971). 
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THE BRITISH OPEN UNIVERSITY: CONCEPTS & REALITIES 



Brian Lewis 
The Open University 
Institute of Educatlormi Technology 
Buckham^ire, En^and 



PART t - TEACHINC AT THE (N»EN UNIVERSITY 

Inirockictory Remarkt 

The Open University was estalriiihed (by Royal Charter) in 1969, with a ii^rmit to start teaching in J^uNSry 1971 . 
This may not sound loo difficult. However, no conyenttonai British University had fiver t^en set up so quickty. And the 
Open University w^ certainly not conventional. 

The £^n University is an ambitious project. Its tJeclared aim is to bring hi^r i^iucaticHi witfiin the reach of 
many thousands of adults vtoo are able to study only m a part tinw basis, and in their own homes. Moreover, it is 
striving to do this for adults who have mne of the usual entry quaNficattons- And it is relying for its success on the 
novel deployment of a wide range of educational ^d mass media recources 

In short, the Open Unlve^sit^ is the first attempt, in Britain, to set up a full-scale home-bas^, multi-media 
system for higher education* As the original government committi?e put it: 

"the object of the Open University are to provide opportunities, at both undergr^uate and 
postgraduate level, of higher education to all those who for any reason, have been or are being 
precluded from achieving their aims throu^ an existing institution of hi^e*' educaticm^ This 
does nor imply competition with existing institutions, but rather an attempt on a national 
scale to complement their efforts; ^ attempt which may well increase the demands ufwi 
existing institutions, as students, stimulated by the exfwrience of part-time study, increasingly 
come to want the opportunity for full-time study/' 

At the time of its cont^ption, the (proposed) Open University had no close prece<tents anywhere in the 
world. Nobody had any clear idea of how m»iy students mi^t want to enroll, or what they would be like, or 
what their abilities mi^t be. The one attempt to forecast possible demand came from a sun^y canicd out by the 
Ni»*.ional Institute of Adult Education, This came up with a forecast ranging from 34,(XX) - 150,0(X) in respect of 
those r^ople who said, "I will certainly be one of the fin^t stuctents/' 

For financial and administrative re^ons, it was necessary to restrict the sire of the first intake of students-- 
with the result that 24,0CK3 students were provisionally registered (in 1970} and 20,(X)0 of these mrtually paid 
their fees and started woik in January 1971, By the close of the year as mi»iy as M% had stayed the course, and 
75% of these passed their examinations. This pattern of sucass was repeated in 1972, and the public image of 
the Open University is now gratifyingly hi^. At the time of writing this overview {September 1973)^ the Open 
University has just held its first graduation ceremony in respect of 9(XJ graduate. The current stiKtent population 
stands at around 40,000. And owr 35,(M)0 applications have alre^y teen received in respect of courses due to 
commence in January 1974. The prevailing mood within the Open University is one of cautious self-conyatulation. 
Everybody recognizes that the system is far from perfect. But the political decision to give it a chance seems to have 
t>een well vindicdtcd. 
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.Ava»fi**ty imlicanst/the 0|wn Un»»ie»si|y iiiHiitMrify aliome hased «tudV f"^^ The primafy leadiing 




\. fVtf f at riacka<|» of honm* stt^ materials 



2. NatftmwHfc^ td 




Th*rMt^^a«tVc^l€,s pfowrfe oppofttmitksfwstot^ theiriirotjlems either (ajamoo^ 

in host .mivtff MiM^ .Midbte StwJwtts <^ in ia<temm^ aiWt fo? irore ainiiliimis fJa*v««: A va^ 
of artfiittofta* tffdtrhimi a«fc - «ich m Hortw ei^imentai ki% comfxiter termihais. s«>^iKfcniH»t6««d compretWfmibn 
ffSK. iH>fTwwof k aiid fiekhvtj^k assignments, tutor^narked araf cwi^uter-marked fee(»MBk on Homewwtt. amt 
assorted mvhu visiM aids all help rtie student i6 coiisoiidale his leWojug wd to monit^K hH progress <n an 
accurate viav . . -.• : 

To ^»taiii a; ri«^ at the C^^^l^ 
otdmai y ^tegif^. mi 8 "courie creilits" %r an h<wouf* degree, to do thts» tfw siwient must «ucc«S«vf ty cqmt>»etB 
frS cmiisW ai »f u«tv, at A rate of hot mora than 2 courses per year, ^adh Courte CaHs fof gbom %^ hours of home 
»t»R<v tm w«e»(, siKtainetf over a period of ahout 9 mmihs of Afe v*af . Sft a« »tu<tents are atile to Cumulate the 
fwpiirifrt n«nih«^r of credits in their siMte time/lf tl^V So d«sk«. 

Ir* its f.rif Vear of teaching (197 T|. the OM twttiated "'teuodation" (Urst year) courses Jn the At», Soda! 
Sriertir^. Math«fmatics and Sdeno!. tn its secofld year uf teaching. * first year fcnindation course wa» also started 
in Tiichruiioqy , ^long with stjcond lewel cour^ in Arts. Social Science, ^hematics, Sciem» and Educaf ionat 
Sturlips. Six ctJwse credits secure a general (ft A,| degree, and e^t course cfeditt secure an honours <t.A.» degree. 
In renain ca^, stuilents can be ijr^ted evainption Uom 1-3 of tWe lower^evel courses. wkI this redM^ the time 
neeclwt to t^hfy It therefore takes about 3-5 ytars for nwst stuctents to retain a <kgree, althoui^ ewy effort i« 
bflitiii matte to Mio^ slower students to omtnikite therr degr^ at a slower rate (e.f . one «»urse per ^r} if ttey 



Professional training and "up-dating" courses are also l»ing fim ~ e.g. in Edt«ati<m, Computins. ami m 
varitnis brarwrhes of Technology. Tlw first of the^ ha«e Hr^y started, mvA later wies will tie rtetennln^ 
public «Jemand ^d available resources, T»wy mH be of shorter duration (ret^iirins, say, the ^umuf ation €>f only 
1-3 c(Hin» c»editsK and vwit lead to the award of certificates or diplomas. A p^tspradjiate i»-o{pamme, i^irq to the 
award of higher degrees, has also been launched. In all cases, the OU ain» to i»e esstntially the same kind o» "teach- 
ing at a (^stance" mulfi -media af^oadi. 

It IS worth addinq that the Open University's cmirses are in ipen^^al rfesi^piefi to have p^M:|icat airplicatiwi in 
the r»»l world. They are seekii^ to meet genuine social and economic needs, ami they should therefore be of 
tiractical value both to the student *id to potential emptoyers. 

Vtho tfie StiNfents Ate 

As already iodicatetl, the OU is first and foremost a Uninerstty for »kilts. In the years to crMne, it may becwne 
Iwth posviNf" aiul desiraljle io accept ordmary sdiod leatfl&rs in (say) the 18-21 age rwige. How^r, the initial 
dpcision was to »imit n»»body under 21, unless there wei« exceptional circumstartces (e.g. of f^ysical dis^ity) 
whicti would iJftfvan a innson umfer 21 from ^aininq acceptance elsewhere. At the request of the Department of 



«Mf«sire. 
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Education ard Science, the University is currently planning to «K:c^t 500 18^ye?r otds as a piJot experiment <n 
1974, 2M) of these %vi»l t^' tet^nicatly qudliff^ for University entry, and 2% will not- 

in its efforts to offer hi^er educalic^ to as many ^uhs as possible^ the OU has waived many of the 
traditional entry requirements, it therefore caters for a lar^ number of ^ople who would otherwise have bet*h 
prevented from pursuing degree courses. These tnctiKle ^ultr in fuii-time employmt^nt, houS€swivt?s who find 
it hard to ieave their hom^, per«>ns w^odo not hai^ the usually-mandatory educational qualifications, persons 
wno are physically handicapp^, and persons who live too f?r away from already-existing college of higgler 
education. Among tftt fully-employed part^tirr^ students, there is already a si^ificant number of adults who 
are looking to the Open University to provide qualifications for an alternative career, or {h) to prov 'Me 
kinds of specialist up-datinp and retraining courses that have already b^n mentioned. 

The implications of such a radical "open door" (Mlicy should rtot be overlooKed, It givt^ rise to a hi^ity 
het**rc^n«ous population of students — of varying a^ lewis and interests and backgrounds — and adds greatly 
to the complications of teaching effectively at a distaice. By way of example, diagram 1 gives somtf summary 
background data in respect of the 6,790 students who studied the first year foundation couf se m scit^ice during 
1971. 

What the Study Maferiah Look Like 

Study materials are $ent throuj^i the post to the student, at intervals of about 4-6 weeks- Each ^ackaiit* 
contains a sequence of corr ^M^nndence materials, accompanied by study notes, exercis*is and ejcperiments, and 
self-administered comprehension te^s which the student can take to help satisfy himself that he has umlerstood 
the matn teaching points. Also mchnled is a set of written homework assignments which the student is expt?ct«»d 
to f$tuw, wuhin a specified time 9er\c^, for marking. Some of these assignments are marked directly by the OU 
computer, and others are marked by specially-appointed part-time corresponav»nce tutors. 

The "core" correspondence materials are (usually) specially written by mer»ibers of the OU's rentr^l 
academic staff Tht? remaining materials consist of carefully-chosen extracts and of »pnnts, taken from standard 
textbooks arid other professional literature. Each package also contains discussion noiC5 (which the sturit^nts art* 
invited to discuss among themselves), and recommendations concerning suitable follow up ft:^ding. Additional 
materials in * ie form of auoio tapes, long-playing records, and transparencies are included \n iome study packa^jes 

Most study materials make a special |K>int of giving the student a variety of things to cA). Tasks are set which 
can take the form of pencil and paper tests and exercises, scheduled observations, group discussions, home 
experiments, local fieldwork, and the like. The aim of such tasks is to create experiences in the student which 
help to confirm and consolidate the expositions contained in the ririnted texts. In particular, all science and 
technology students receive a comprehensive home experimental kit whidi requires controlled and systematic u^o 
and reportage. 

So faf as is poss'blf*, each study package is a self-contained entity, ft provides the student w»th everything 
that ho needs, m order to advance ht$ studies for a further 4-6 weeks. In some courses, it Has been fouml nect^^t^^ry 
to refer the student to "set books" that he is expected to buy or borrow from hfs Utnary . But thn stuc*y pacH«i<jo 
constitute in (^nprsi, he core materials of the course, 

The Role of the British Broadcasting Corporatlcm - 
an Education^ l^rtnen^hip 

To reinforce and supplement the packaged study materials, th*^ University works irt collaboration with the 
B.B.C. to produce a regular sori^ of radio and television programme. At the present timi?, inanities exist for the 
transmission of one radio and one television program per week, m lespcct of almost every mam course th^it 
the OU is f)lanninii to run Transmissiorn occur mostly at off-peak iistcninq and v^rwrnq times. Each prcqram is 
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Summary data on CKxupation, ^ucationat 
back^iH^, and age of the total intake of 
6,7OT students «fho studied first-y^ar 
&:fencein 1971. 




Occupation 


% 


Housmriim 


4.8 


Armed Fonrn^ 


2.3 


Admni. and rrwiagers 


8.0 


Education 


26.S 


Profi^tor^l ar^ mu 


7.2 


Sc^tists arKi engs. 


11.6 


Technical penKmnet 


23.6 


Pitied trades 


2.5 


Other ms^uai 


1.6 


ComnHinic'n and tiamport 


0.9 


Ciericai and office 


3.0 


and personal serv. 


3.2 


Retired/ftot workif^ 

— — — — — ^ — 


1.1 


^, ... 

Educational 




% 


level attained 




No formal quals. 


6.6 


CSE-RSACert. 


1.6 


GCE "0" level. l-4Subj. 


6.9 


GCE "O" level, 5+ Subf. 


11.3 


GCE A level. 1 Subj. 


3.5 


GCE A le^l, 2+ Sub}^ 


9.7 






HNCor HND 


19.3 


Teadier's Cartificate 


18.7 


Uniwrsity Diploma 


8.2 


University C^gr^ 


4.5 


Date of birth 


% 


Post 1345 


10,8 


1936-1945 


44.4 


19261935 


30.0 


1916-1925 


12.2 


1906-1915 


2.1 


Pre 1906 


0.4 
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broadest tmcx - once in the «ai1y evening of a vi^lcday, once cm a vveek*^ fnomif^. So ewry stiKtent tm 
two opportuniiiiis to listen view. This is a u^stantiai unt^rtaktng v^idh p^dy enhitfioes the effectiwne» of 
the wrinen study materials. However, it will mit be nossifaie to rqaeat (»^0|pams in future y^rs unte^ more riKfio 
mni tiHeviston channels L»a)me avaiiaMe. Am) ptons are ^ready unctor way for "dawn" list^ine viewing. 

Radio is used in a variety of ways — to artmtate the stm^t towa-ck stmJics that he is }ust atKxit to ocmnnenoe, 
to discuss suhiects that are inacfeK|uately dealt with in tfie c<Kresp<mdefice n^tertals, to i^capittilate or summarise 
n^teriirfs that ha^ aireaif y been studied, and so cm. fUdio is alM ami to communicate with stuctents at short 
notice - e.g, to clear up recurring (K>ints of diff icutty^ or to nuke anncnjncements of interest to certam sections of 
the student population. OccasionaM' arrangements have bem made for nation viride hooknips between central 
administrative and academic staff, re^<Hial s^ff, aiKl volunt^r stiKlent spcAesmen, 

Television can also have a vallate orientating function, esi»ciatly at the beginning a cc^irse when ttw stu- 
dent is striding to nake s«n^ of unfamiliar concepts. Itovw^er. it is most commonly used, at die fre^t time, to 
convey information tiiat requires special vt^l effects. Lecture and demonstraticms whidi invcrfve ttw of 
special (e.g. scientif tc) equipment, smalt ff'oup discussiors on natters of controversy, visual ^:ooints of the Arte 
anl Sciences, of ^veloping technology, of pcdfttcM Btftd eccmimic and educdtional ^stenn, of unfmniliar coim- 
tri« and societies and civilisatitxis, of our ovw {»kI other |:m>ple's) historical and cultural heritage - these are just 
a few of the subfect matter areas to whidi television has usefully been apfriied. SiMne quite distir^idie^ and highly 
motivating "evocative" f^o^-ams haw also tmn made. Television, tike radio, h^ th<s c^:^ty to l^ing education 
ri^t into the student's own home. Correctly twed, these media can do much to add a dynamic and personal flavour 
to the tuitiw, and they help to fostor a of participation and involvenwit in a major national vwture* 

Ttiton, Cmimetlofs md Study Centres 

To provitte mc^e personal supj^rt for thc^ v^o f^l the need for it, the OU has set up nearly 3(K> study 
centres throu^out the country. These typically a»isist of a small numbei of rooms that sympathetic authorities 
Oocal colleges of education, for instants) have hired out to the Univ^^ity for purposes of study wd dfscusskni; 
AH centre? are equipped with radio and television receivers. Many of tt^em also contain tape reorders, projectors, 
a library of the broacfcast n^tertal in recorded form^ and coinputer terminals for the %m of matfrematfcs students. 
Each centre is reg^it^^ty visited hy ^fwially -appoint^ Course Tutcws and Ck)un$eIlors, The^ are profe^tonaily 
quatif red people (lecturers from nearby colleges, for e^amplelwho are appointed wi a fwrt^ime basis to hc^d 
seminars and discussion ^oups, arnl to advise on the variot^ i^irfjlems - both academic and persmial — that 
individual students might be encountering. Every effort is made to keep study <»ntres open at times {throus^out 
the evening, for example) which are nK>st convenient to all ccMicerned. Discussion grtnips and tfie Mke are ^so 
held at these times, The whole stkwly centre operation is KK>rdinatod md monittKed by Senior Tutcm and Senior 
Counsellons who are full-time members of the Ojwn University Staff. These senior staff are b^ed on Re^onal 
Offices but they lernJ to travel around a Qteai deal and hence provide critical links between the local part-time 
staff and the central staff at University's itiain headquarters. 

The number of part-time staff associated with any one study centre isriof, of course, fargp. But every stu- 
dent fias several chances every month to disciftS his problems - eitfter individi^lly, or in small group discussion - 
with the appropriate part-time member of staff. If a stu(tent is unable for some reason to attend a study centre, 
special arrangements can be made to visit that stuctent (albeit infrequently) in his own home. Part-time staff there^ 
fore operate rather like ritxrtors. They have a caseload of students. At certain times of the wwJk they are out visit- 
ing "emergency" cases. At other times of the week they are available for con&uUoiion at a study centre. 

Attendance at a study centre is advisM^le. but not compulsory. At the preent time, study centres are locate! 
mainly in well populated ireas, so that the majority of students are always within a few miles of such a centre. As 
indicated above, many study centres consist of rooms hii^ in some nearby college of ht^er or further euuu*lion. 
Tht<i Hicidentdfly bnmp OU students into contact with other student bodies; and it may eventually toad to an 
extension of study centre facilities. If the demand arises and if resources permit, efforts will be made to enrich 
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the OU's study centrcii - e.g. by ^ing library bock**, t^ratory equipment, and other learning resources. 
Additions of thjs kimi wiH t)e east^ to make if ttw study centra are already homed within educational centres 
that sympathise w«ih thi; OU's <tenef al aims and <riifectives. I 

Resdentiai Summer SdHMrts \ 

Vt^ educational resources so far de^ibed (CDrr^ioraieni^ materials, hc»ne kits, r^io and television 
pro(Kammes. anti study centre discussions) are intemled to be sufficient, by themselv<»/to (^orrwte ef fwtive 
learning. Even so, a case can tie m^ for getting stucfents t<^nter in imich groups {a) to convey ^imething 
of the trailitional "univefSity ex^jerience'% and <b) to achieve certain acklitional obj^iws that mitfit otherwise 
be rHH^ected. The OU fws acc<wdin|^y ileviseti s system of one^v^ resident lat simnw »:hools, v^tdi are hdd 
at host Untversilies in different parts of the country. Summ^ scfK>ols are compuls^Ky and ttiey are currently 
being arri^n^ to take place during tfie ^mm^ vacation of the host Uniwrsities invd!^. Since OU's 
academic yeaf runs from January to early October, sumiv^ «^)ools therefore ocoir about twvo thinJs of the way 
throu^ each studefifs course. 

Each one week summer school caters for several hurrired students drawn from different parts of the OHjn- 
tf y. Although the per ioti of residefKe is sfrort, several noteworthy objectives ran be achieved. For eKamfrie, OU 
students have the chance to use and savour the host Unim^sity's fwilities - fuity-equt|;H)ed laboratories and 
ilemonstraticm rooms, large scale lecture halls, reference libraries, and the like. Central academic staff also have 
an opportunity to meet students ff/? /riassff and individually, and this can jKovide usehil additional feedback on 
stuilent attitudes and progress. 

Summer schools can serve a variety of useful purposes. Tf^y can provicte <w>ortunittes for intensive 
laboratory work, usmg a full range of standard equipment. They tan provic^ opportunities for special project 
work in {for example) the Social Sciences. They enable large scatr^xhibitions to bei^t upi^r^ visits to be 
arranged to places of cultural or scientific interest. They provide a forurrvior mass discussion on^natters of 
difficulty or dissension. They enable distinguished visitors to giv^ Invited addres^s. They provide oppQrtuniti^ 
for setting and discussing and evaluating special homework assignments. They provide opportunities to re run radio 
and television proyrainn>**s that students nuy hve missed. Ar^\hey provide aikiitk^nal opportunities for members 
of staff to engage in diagnostic and remedial teaching, / 

Above all, courses of this kiruf enaWe students to get toknow each other, ancJ to exchange views and ex- 
periences. This can have a slroiig motivationaf and integrative effect on the student population as a whole. And it 
can help to counteract any tenffencies {induced, perhaps, by the study rantresj for the student pwulation to 
f ractHM^ati* mto small inward-lookinq groups. Similar comments apply, of course, to the numerous members of 
staff ticademic and administrative, full-time and part-time central and regional - who attend the varirus schools 
that are run. These members of staff are able to meet each other, and to meet a wide cross section of the rtudent 
population, all at first hand. 

Evidence from other countries suggests that attendance at a summer school, even for a short period of time, 
can have a notable tnlucalive effect. It can also boost stuttent morale {even if the stiidents spend most of their 
siMre time gcHsipini] and grumblHig). and it can help to promote a sense of corporate identity . Home study is, at 
the best of times, hard going. The rallying effect of a welhrun summer school should not be underestimated. 

Students as Teaching Resources 

No account of the University's teaching resources would be complete without some lefcrence to the stu- 
dents themselves Most students tend to be regarded solely as learners - often as passive and inadequate learners 
who ideally nee<l to be c^rtifuMy nurtureti by expert teachers and great miiKls, In reality, students are a fjotent and 
much-nofjlecttni teaching resource. If a University's formal teaching resources {its lectures, set books, and the like) 
aie in .iny way def<»ctivf^ ^tM\ if no t(>uchHr i$ around, students are obliged \ j do thfi teacher's job for him. They. 
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must acquire the skill of extriKrting information from books and journats and from eadi otfier, ami they must use 
e^h other as sounding iKidrds for their own deveii^ing c^incms mni expertise. 



The most successhil students all tend to pick up these self-help skills at a fairly early sta^. At the OU. study 
centres uridoubtedly help to foster the developnwnt of sik^ skills. By m^ns of snr^li-group disciHStons, stuttents 
are more able to discover and qwstion hittwto urwecocpiised assumptions in the subject matter they are studying, 
and to uncover gaps and limitations in their own knowledge. They also have the diance to pursue lines of enquiry 
which are of special interest to them, but whkh are dealt with only li^tly in tlw n^in stiKly materials. 

One of the best vays of iedm^g about a new subject is to try teacfiing it to other peo|de. There are, of 
course, sonw cAvious dangers in the Vnethod. If nobody in the group knows what he is talkir^ ^out« group dis* 
cussion will m^ely produce a sort of\"pooled iffiorance". Oi the o^er hand, carefully i^epared study materials, 
duly backed up with annotated discussion notes and easily-»x»ssible foti<WHip reading, can arable students to 
hold profitable small-group discu^ior4 even when no tutor or <x>un^lor is present. Students are a ^chif^ 
resource in the sense that they (a) amipensate for (teficiencies in the iiesoufces provided by the University, and 
ib) help other students to extend and ipply their newly-2«:quired knowledge in accortiance with the needs and 
interests and abilities of those students.\Corsiderable effon is being n^de to ensure that the study centres fxovide 
a suitable environment for the conduct of such activities. Failing this, students are being encoura^ to set up 
self-help groups in their homes or at other suitable venues. 

Organisational structure of the University 

The headquarters of the Open Univerr;tv are tjeing developed in a rural setting, some 50 miles north-west of 
London, The exact site is on the outskirts of the ctesigpriated area of tt^ new city of Milton Keynes, in North 
Buckinghamshire. It is dose to the Ml (Loreion-ScotlfcrKJ) motorway, about 4 miles north-east of Bletchtey, 

Modern building, which have all been specially erected, are currently housing several huralred academic 
and administrative staff - together with a large-sized data processii^ ami record-keepjng system, and extensive 
facilities for the design and production and packaging and dispatdi of the Uni\«sity's study materials. Also 
nearmg completion are research laboratories (for jarademk: members of the staff), a much enlarged wd lending 
and reference library, a lecture hall, canteens, and so on. There are no undergraduates on site, but room is 
currently being found for a small number of postgraduate students. 

To control and monrtor a nation-wide home study system, a rather elaborate organisational structure is 
required. For this reason, the University has divided the country into 13 samiautc^omous regions. Each region 
has Its own Re^onal Head Office, staffed by a Regional Dinsctor amJ other senior administrative and academic 
personnel (in particular, the senior tutors and senior counsellors mentiorred earlier). Each Regional Office main- 
tains detailed records of all students comir^ within that region, and the Regional Office staff are responsible for 
coordinating all the University's activities (tutorial, counselling, study centre, summer school, etc.) that go on in 
that region. The overall orgaofsational scheme is partly captured in diagrams 2 and 3, 

The formal structure of the Open University is therefore hierarchical. Part-time tutors and counsellors are 
responsible to the Regional Office staff who are in turn responsible to the University's headquarters at Milton 
Keynes. In practice, of Course, the real day -by-day structure is heterarchicil, Ojntrol over the University's affairs 
fluctuates around the system and, ideally, always tends to reside with the person (or group of pei^ons) best quali- 
fied to act. 

The Assessment of Student Performancer 

The OU has departed f ^om the traditional practice i'm British Universities) of relying almost exclusively on 
formal end-of -year endminat^ons. It has also tried to recognise some of the many limitations of traditional 
numerical marking schemes. And it has taken account of the nmls of adults who are struggling to study at home 



199 



ERIC 





opf.« yw^i-Ksnr HQ A1 vV.^ltom hall 



ACADCMk' f >.f f 




AMuaN».'^:ii$' 



tOCALiiryoYcfta.'.: 



COi»f.SE Ttircf.; 



coirwsitiOH 



r3dici ami ttlfvrtto.-i 6<>t4- 



1 



BCC ? BHOADCASTS 



bxpcfi'moht for 
r.(;<.<!»fM:c- m<{ 
1 ; chiic'es.'/ f.n-: 



STlfOENt 



RESJOENTIAL 
SUMMf H S<;HrOL 



BOOKS 



set bCK»t s 

LfSRAHiES 
bock^rouiid (vading 



200 



m th^u 2»pjro timtv .iiwl ^vho rii«»fvfoft» feci the neeil for rvQular fothlback oii their proijross. In the liqtn of these 
considerations, the OU fus clnisen fnitially to ofK?rate a "continuous assessment" prM^lufe, ushh| {in rmjst 
<!jse%) jt/rot/fs tjthei th.in fUiint^ru .il iii.iiks. 

As dlteaily indirdteii. j|l study itackji^'s cofitain hon^cKk assi^ments wf<ich students areex^^ectetl to 
fetuf n, duly comfiletetf ami wdhin specified time limits, for marking. Some of these assi^ments an? in computer- 
mjrkalHe form, ami students jre recHiesteif to f>ost these directly to the OU's head<iuarter^ at M Iton Keynes. The 
renwimmj assujnments are sent to the special group of part time staff mentioned earlier namely, the local 
correspomiena! tutors. 

Performance wi homework assignments is recorded, for each stuc*ent separately, cm computerised stut^t 
record files. If a student has umfertaken any special work alignments (e.g, at stutly centres, or summer schools), 
then the marks oljtainetl for th<?se special assi^ments are likewise recorded ir lividually. At the end of each study 
course, all students are require<l to attend an Examination Centre to take a short examination (of about 2-3 hours 
duration) umJer properly supervised conditions. IMarks obtainmf at this examination are then compareil and 
combined with all the other marks that the student has accumulated, over the year. Finally, a weighted composite 
mark is calculated to decide whether a course credit should be awarded,and, if so. whether it shouM be awarded 
"with distinctKwi". 



It IS worth twarimj in mind that the continuous asse^ment procedure, toother with the week -by-week flow 
of essential radi') and television programmes, can have a powerful and son-.etimes Of^nessive pacing effect on the 
stiKient population. The inifHwing of time limits on homework, and the study sequence of syrKrhronised B.B.C. 
broa<1casts, constitute a strong indticenwnt to students to keep up to date with their studies. In recognition of 
the fact that some students may fall behind with their work for reasons that are beycmd their control, all students 
are allowed to skip a small number of assignments if they so desire. In effect, this means that the award of the 
overall continuous assessment grading is baseil on (say) the best half^ozen assignments that each student sends in. 
For an elaboration of the implications of this arrangement, see Lewis (1972), 

The Handling of Assessment Data 

Omiputer markeii assignments consist of multiple choice (aruJ similar) questions that can be answered simply 
by makinq a seot i^ of marks on specially designed answer sheets. Students post these answer sheets, duly completed, 
to the University's hftadquarters. They are then decocfed by an automatic document reader, and evaluated by 
computer . 

Tutor marked assignments corisist of essay questions and short answer questions which, by their very nature, 
cannot be markeii by computer, Sturems must therefore post these assignments to designate! correspondence 
tutors, who are responsiijie for marking them in accordance with pre arranged guidelines laid down by the Univer- 
sity's central academic staff. 

For each homt^work assignm(»nt that is received, the computer generates a detailed statement of how well 
the student has done la) on individual questions, and (b) on the assignment as a whole. At the present time, the 
OU computer has only limited facilities for interpreting ttie homework data that it receives However, a variety of 
computer programmes are currently being cieveloped to enable student progress to tye evaluateii with greater 
s«?nsitivjty - e.g. by showing whicn questions, and which groups of questions, cause the most difficulty to wtiich 
kinds of student. At the level of the individual student, cumulative indices of success and failure are also being 
devised to show how well each student is doing (in comparison with other students, and in comparison with his 
own pust performance) at each stage of the cours*?. It will then be easier to defect those sections of the course on 
which the student does best (or worst), and also to detect any su<lden improvements or <leteriorations in imlividual 
fH?rformanre. If successive course ur)its tend to vary unevenly in content or level of difficulty, this should also be 
more easily detectable. 
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From time to time, the computer ^erates summary statements - both to Regional Offices and to ttie 
student themselves - showing how eadi student is (»^<^re$$mg. If any student shcwvs evidence of sudden and 
sev^e detenoratitm, emer^ncy signals are sent to the Regional Office, so that the stuitent in c^ie^ion can tie 
contacted ami offertHJ heJp. 

Similar initiatives can be taken by the correspondence tutors, and by all other part-time merntiers of staff: 
As a matter of routine, the correspondwce tutor posts the student's homework t>ack to him, duly marked and 
annotated, b& quickly as po^ibkt At the same time, he sends a record of his marking (and any further comments) 
both to the OU's headqi^rters and to the Regtcml Office, The tutors who attend study centres, and the coun- 
sellors who also attend study centres, can likewise report back on the i^c^ss of individual students. It follows 
that both the Regional Office and the OU's headquarters can dK:cumiriate a large amount of information at)Out 
student pro^^s - information which flows in from a variety of sources, and which needs to be sorted and inter- 
preted by the kind of software now under ctevelopment. 

The OU as a Sdf -improving System 

When students return their homeworl^ assignments, they are periodically invited to retpster their opinions on 
such matters as the inleJligibility. difficulty, and interest of the assi^nwits they have completed. They are also 
invited to say whether, and to what extern, they have made use of the OU's study centre facilities. At tt>e same 
time, the part time members of staff - the correspondence tutors, the tutors who attend study centra, arKi the 
counsellors - are asked to cf?mment on the students that come under their care, and on the study course as a 
whole. In particular, part ^ me numbers of staff encourage students to talk about their academic and personal 
study problems, so that the OU can build up a more accurate picture of student needs and interests. 

To inducp feedback of this kind and to secure more specific types of infom^tion, the University has esta- 
blisheii a Survey Research Department which regularly sends questioi^naires to representatiw samples of stuctents- 
Eventually, it is hoped to set up consultative panels of students, and to have fuHy trained interviewers and social 
workers circulating around the system. There is, in fact, already provision in the Royal Charter for the convening 
of Regional Assemblies ami consultative committees and for student representation on the Senate, Arrangements 
have also been recently completed for the starting of student magazines and newsletters. 

By way of summary, dia^am 4 records the o^in assessment and evaluation procedures currently in use at 
the Open Unfversity. As can be sef^n, the data collected from computer marked assignments (CM As) and from 
tutor markmi assi9nments (TMAs) aixl from various additional sources (eiul-of-course examinations, question- 
naires, letters, telephone calls, etc.) can serve several different purposes. For example, the data can: 

(a) provide the main basis for a formal assessment of student achievement, and 

(b) enable relevant feedback to t3e given to individual students while the course is actually in pro- 
qn?ss. and 

(c) facilitate the revising of those parts of the course that appear to be v ling less effectively than 
was oriqtnaHy hoped. 

With regard to (c), it \z worth remarking that each course that the OU runs is carefully monitored, during 
the first year of two of its existence, by a specially -appomtcd ''maintenanco feam" of academics. This team is 
responsible tor modifying parts of the course in the lit^t of the total assessment and evaluation data it receives. 
Unfortunately, the current budget for revision is severely limited, and tf»€ revision process itself is largely a 
judyniamai one. (In other words, it is not guided by any cogent theoretical model). Diagram 5 depicts the main 
channels of communication involved and, collectively, they seem to l^e more than sufficient to guarantee that all 
fT^/or defects are picked up during the first year of each course's presentation. 

Although the student pu^ulation is scattered across the whole country, the numerous communication 

chanru'lh sho^J'jn m HhKjt jms 7. 3 .ifnl 5. miki^ it companitiv^^tv ojsy to qiither a qroat deal of information on 
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dCddemic and personal study probttmis, ami on the strengths and weaknesses of the sttKfy materials, on the effect- 
iveness of (he tutorial &nd counselling and sti^iy centre facilities, and on the condrtiom under which the students 
are most likely to succeed or fail . Data obtained in this way are helping the OU to sirmgthen its instructional 
system slowly but surely, from year to year. 

In this connection, it is worth noticing that the OU has many of the dwfiKrteristics of a Direct Action 
Research Project, instead of opting for ttnHs-consuming '^fe^ifoiltty studies'' cmi muHi-media systems of higher 
education, the Brittsh Government to<^ the imaginatiye step of dually allowing such a system to be ^t up and 
run. The result was a hi^ risk undert dining in which mistakes vwre bound to occur and to ^ on occurring. But, 
in the long run, this could well be the quickest and frost effective way of solving the many intricate pf ofotems 
invcrtved. 

PART 2 - SOm PROBLEMS OF COURSE PRODl^TiON 

Initial Conceptiom 

in the earliest days of the OU, before course production actually began, most xademics were fairly optimis- 
tic about the ease with w^ich courses couhJ be (»^CKiua»J. For example, ft was widely believed that, during the 
first year of course ^oduction (1970), a group of 8 ik 9 academics - along with a handful of ex^rt ^fvisers and 
assistants - would have little difficulty in |;^oducii^ a full ^-week Foundation Course. It w^ recogni^ that a 
great deal of written material would ha^ to be produced. But there was a tenctency to f^ceive this particular task 
as requiring little more thiKi ihe ima^native bri^tening up of already eKistir^ lecture notes and professional 
papers. The whole operation was one tfiat could be contemplated, in advance, with eqt^nimtty. 

irKleed, many academics confidently expected the business of course production to occupy not more than 
1(K)-2(X} days of e^h calendar yejr» Universities char^ertstically have toth a tea<^ing function and a knowied^ 
extending function. The former is concerned with introducing stuctents to knowledge thai already exists. The 
latter is concerned with challenging ami deepenifHi aod enriching (and even trwisforming) such knowledge. In 
accordance with this dual image of what a University should be, most ^demies took up Xheir appointments on 
the explicit understanding thai they would have time for at least some of Oie more sijpiificant knowledge-extending 
activities - postgraduate supervision, private research, professional writing, consulting, conference attendir^, and 
the like. In other words, the OU was to be a real University, not just a production house for the devetopnnent and 
dissemination of undergraduate course materials. 

The Harsh Reality 

These expectations seemed reasonable enough at the time. But they were rapidly shown to be sadly inacctirate. 
In the first place, nobody had fully thought throu^ the numerous implications of requiring ac^mic members of 
staff to work together in lar^ cooperative teams. Secondly, no-one had fully pursued the imi^ications of having to 
*each a heterogeneous population of students who were mostly trying to study in their si^re time land perhaps for 
'.he first time) in the isolation of their own homes. To cut a long story short, it s<xin became obvious that in order 
(a) to produce effective teachmg-at a distance materials, and |b) to have time to spare for the various other 
activities m which academics normally engage, the overall staffing of the OU vwuld ideally have to be iitcreased 
by a factor of at least 3-4, This kmd of increase was out of the question, becai^e the OU budget would not permit 
it. So a number of compromises and sacrifices had to be made. 

In an attempt to establish high teaching standards with too little staff, both the academic and non-academic 
staff proved to be extraordinarily self-s«:rificing. Many academics have, for example, forgone substantial periods 
of holiday and study leave due to them. They have cut their private research activities to the barest minimum, in 
order to devote themselves almost exclusively to the business of course production. At the same time, the rate of 
production of nf*w courses has been sliqhtly diminishec', so that the courses that &re produced stand a better 
chance of doing an effective job of teaching. 
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As a ff»sult of t,ikifi(} tfn?s*? sti;|H, thi? pr mlucfion of ifiulity ctHiru^s is now mefdy iftfficult, rdtfmr than 
*m|H)ssih<i«. But thf coiitjituins 4if stvvici? fur OU ^icademic^ (laiiil for many of tho supporting staff) must in: 
.»MuMv*j tfio wnist in thf rotintty. rivt^i ii stiH an i»\h.ujstffH| iu;h aqain^t timi' to |if<MUit:e hicifi (fudltfy 
nnitt'TMN Ami tfu* ^jn-sHiff tn*iy <:ontifHi«* fut at lc.tst S tO ycais jHfrjusi! this i% tfic tinw thdt it wi!l takt» tu 
i*^t««lilfiih <i Siifnl stcwk i)f vvfll vjlid.ifeii couf Si*s. 

The Siw of the Probtcmi 

It ts <!iff ii:uH for m ixjlsiitef to ainirtjciatu fust how atmor mal are the prcrtjfems th*it confront OU acacfemics 
iMul »niniinistf iiiivf stJff. Thf oljoriimi ami proiUictton of corrt^sponilenct! iraterkils, homeexi^irn^la) kits, stij- 
ck»nf tivitu's. rmihipfi- rUtuiu^ tests, i.Hiio atul t«*leviSion programmes, atuf thu like, make dem^Hls on staff whicfi 
aif ofilik*' dnythinci they haviM»ver ♦•xpenertanl liefore. For exampk?, SfMjcial earn must lie taken to ensure that each 
corresjH)iHlf?ncv unit <s a unmi\ and cot^rent aiuf memorable piece of exposition. The mam teaching fwints must 
inf fully fKplainiHl Misl»Mdmi| stalmnents ami irrelevant scfiolastic displays must tie elimmateil. There must be no 
mistaki fuMiMfiiuitut*., qaT)s, nr oth<?r defects m the ariHinnent. All written nvat^^nals nuist in fact be wdl structurwi 
aiui !>Hf fMspi.irujUiry and pitchei! at the nqht k?vt>l of difficulty. 

The v\rfuMi' (H^t^atiofi <s extremely timeconsuming. If cm acatiiemrc wants to write a really effective pitN:eof 
expiisition one that will either stdfid on its own cm^ serve as essential suppiemaitary reading to a prescribed text- 
tHH>k • It could fasjiy take hm) as much as 2-3 weeks to produce an adequate first draft. His first draft would 
then neeil to be tr^^d out, afont| with some approf^iate evaluative tests, on a representative sample of potential 
students, to see wtu'ther tt achieves what the author wanted it to achieve. Feedback data from these students can 
then tie im^i as a Imsis for revising the ongmal draft, and for polishing it up info more final form. Ideally, the 
tevisiHl ilraft shouUI a/so Uv tested (p;ef^fal>ly on a different ref:^esent3tive group of vc^unteer students) to make 
sure that the M'vision fnocess has been effective, arxl has not inadvertently introduced further defects and 
oi3Scuiitie5 In U\ct, thi* itK! f#>v«^p n^rrsf ryrln <ihmild be re^Teated as many times as is necessary to satisfy the 
author (aiul his colli:iH|ufs) that the correspondence unit really does teach. However, each new cycle can add at 
least 3 wt^'ks tu thi» overall priHUjction time. It need hardly t)e said that ttte OU does not have either the time or 
the MSouTCf s to put Its mstructional materials through the optimal niimtier of (3-4) cycles. However, it is not at 
all uncommon to fintJ 50 tOO hours of academic effort i>eing mvesteff in each singli? hour of student stutiy time. 

In ,Klt1itK)n t{i wntinq th<* basic correspondence unit, the author must devise appropriate homework assign- 
ments. If the hof.ii»work is to hi; marked by computer, he must s|M!cify BMactly the marking sctwme that the 
aimputer musf use. At the very least, ht' must say how many itwrk.' sx.MX t>e aiUk6 for each conect answer, araf 
how many ciedurted for each wrong .inswnr. In *uWitiun, he may call for the award of bonus marks for special 
a)mi>indtK)ns of n ncct .^iiswer. And he may request the computer to p^oyi\(\e him with some detailed informa- 
tion on flifferential ^wttrrns of right and wrong answers. 

If the homework is to !h? markwl by a3fresporM:;ence tutors, the author must provide these tutors with 
a|)pfOpriate giiuielines ami checklists, And he must tell the computer how to handle the marks or grailes that 
these tutors f?v<Mitually award. Since correspomience tutors may often be called upon to exptam of 
difficulty, thi? diiffjoi must ritsu circulatf? atlequate briefings on the overall aims iml objectives of his units. In some 
cases, he may wish to providt? the study centre tutors with special discussion materials. Ami he may want the 
students to perform eKfM^rmimts, or engage in certain kinds of group activity or field VKJrk. All of these activities 
greatly add to the total prmiuction time. And all of them ideally need to tx> put throui^ the test- revise -retest 
cycit? to make sure that they are {a) unambiguously clear, and lb) likely to have the effects intended. 

Th4? ^irepaiation of concomitant radio and television programmes is no U?ss arduous. Administrative arnl 
economii: constiatnts make tt difficult to put these programmes through a test levise retest cycle, so even greater 
ef foils must Uv made to ensure that they are right fifst time. Experience has shown that even the simplest looking 
f)to<Kan)me can idki* ^.everal w<?t*ks to plan and piuduce - especially if it has to be carefully integrated with the 

cof fespcintli fH:** fii^iltv f<i^., or if .inythtf^g uut of tht* firdinary .i s|.iectal setiuenn^ id visual effects) is requiriHl. 
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To c^diii a tiisf iif^iijrtt into ihn |tf at!tfc«it iiffficulties af couise production, tet us comklcn^ a sttuaiion in 
v^ich 8 ^demic membm of staff {call th^ Ai . A^. agree to cDnafoG^f« on the turoductiofi of a 32Hmit 
oiHrse. To &im|jlif y the di^cui^iion, shall fMi^ fiotety on th^ imAitem of pradiK^ng 32 cmt^^Hmcl6m:G units. 
Ami «if€i vhall sut^H)!^ that t$ach acadeniic agrees to write imt 4 of the 32 uniu. As a htr iher ski^ificatmn^ 
%h4i\ assume ttm ^tim to w\tt units 1 4 and A^ agrees to wf itt! units BS, mA A^ agretfs to write units 9^12, 
and so on. 

Aftur i^virt a< vvt^eks of ^uint d^^usskMi on the J»fm arnl otitt^iv^ and content of the <xiurie, the iK^mtemics 
9^ away ami start wf iff tH}. If we xc&p% tfw kinds of tiine csttmdtiis nwntioned earlier, and if we a^umr that mch 
academic writes his units m numertcai ordei; it follows that aftf^r a lapse of a few weeks the first //t ^i&nso drafts 
of units, 1, S, 9, , , will fie neanng omi^tetion. Unfortunat^y* this nati^al-toukingarrangennMnt rates out tfw 
possfliility of testing ttw draft units, in a systematic manr^, cm y^unt^ stuitents. Unit 1 <»n, of cmirte be 
tested. But unit 5 can be meaningfully imed only on students who have worked through units 1-4. And units 
2*4 have mit yet bt»)n writttm. 

This Js a vtjry considerahlo difficulty. If tfw academtc menAers of staff are trying to adiieve ^xid continuity 
from one unit to the ne^t. A? witi ha^ written his umt S ^ a followHCtfi to what he expects to api:^r in units 1-4, 
For exampk!. A) mt^t have satd that his units 1-4 will intfochK^ some ti^ic c^ncefHs m $tdt«stk:di analysts, in 
preparing unit 5, A^ wilt therefore f^l free to build upon tfte statistical knowlec^ that he {relieves Af will in^rt 
in units 1-4. insofar 9% ^ kvs this statistical krrowlec^ fc^ granted, his unit 5 w^il tie iargdy imintefligible to the 
volunteer student who has previously seen oi^y unit 1. (Mfficirfties of this kiitd terKl to increa^ o^ successive 
tmits. Ag, who ha^ tiraveiv acK^eii to do units 39-32, » engafing in a tremtndous act of fait!) if his unit 29 tries to 
by lid upon or sumnidrise, material that 7 different cotfe^ues ha^ not yet written. 

Should A2 send off his completed unit 6 to ttie fttedia Production Oepdrtmerrt, with the r^iuest tfiat it be 
set up and piinted^ If A2 does this, he wilt place A^ ufKier a strong moral obti^ttcHi to work into his remaining 
imits (24) alt the knowledge that unit S has taken for punted. If A^ now refuses to c%>-0|H^ate (datming, for 
example, that A^ mistinderstood what tie, A^ , had undertaken to do), then unit S will be partly unintelligtble 
unless exper^Sfve alterations are made at the galley pr<K>f sta^. AfKi A2 is hkety to feel thst A^ has let him down. 
At the same time A2 will have only 3 unit'; left (r^meiy 6 8) in which to x^commodate to the demands of A^ and 
other colleagues, instead of arranging for units 1, S« 9. . . to be printed without fur th^ delay, ft is tempting to 
statrk them away on a shelf so that they ^n f^ altered, if necessary, at a f&ter date. As more units come off the 
assen*ly line, everyone gets a clearer idea of what his colleagues are trying to say. Earlier units can then be taken 
off tf^ shelf (so the argument ^soe$) and adjusted to f.armpnise with the lat^ ones. 

However, it is a mistake to supixTse that the mutual adjustment of units becon^ easier as more ^ more 
units are produced. Moreover, the retention of units leaves the Media f^CKiuction Def^tm^t with almost nothing 
to do at tfie beginntng of the writing year, mid with almcM everything to do at the end of the year. To provide 
the Media PrmJucTt'jni Department with a ste<Niy flow of work« each academic may have to finalise his fimt unit 
before he has ev^ started to write detailed in extmso drafts of his follow-up units. 

In practice, the situation often seems to less serious than it really is For e^rample, it is o^^ to each 
academic to d^h off rough outline drafts (of 1,000 words or so) of all 4 units, <ind to cinrufats thaw among his 7 
ooHea^es. This can tie a somewhat unrwrving proc^urr, liecause ir i equires everyone to display imterials which, 
because of their rough and reai/y form, MHjid not norn^ify be allowed to ^ the li^i of day. It Is, moreover, 
difficult to respond to such material m a a>nstructtve way. There terKis to l>e rather a lot of it. Ar^l, becau^ of 
its tentative natme harsh or detailed cntic«m alv^ys seems to be out of pl^e. There is patently nothing 'final' 
^ut the rou^ outlines product - arwl everyorte tends to think that, when the tin^ comes, he will have vwy 
httte dtfft<ttil* V m *i4Tommod3t(ng to thr wishe<i of his cnM'Mgiie*^^ Unfnrfwnately, th^e hows are rarely justifierl, 
Wfien the time a)mes, it rnay be very difficult lOiieed to adjust to one's cotiea^es in a ^tisfactory way. In some 
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Ciises, an dcaik^mc nniy «?Vf»n itiu? his first outiiitt* tirdf t us •t!vtil«ncc' that certain proiiosed adfusi mi^nts are mitea^ 
sondtilc. "If you wamtHl nu* to incurpordtu that sort of tfiaterial into my unit, you shoifid hdw? made your 
n*f)iiifiHtiifttts dfaf <f wtH!hs iiijo. . . 

This kimi of comment «i nwde in fM.ffectly qixni failh. At tha root of all these clif ficiiitie^ is the failure to 
cimimunirati; intentions umj rijquirennints. However wetlmeaning and coi^ative the 8 ^ademics mh^t be, 
they habitually fail to s*?e, m the first imtaiHre, the foice of the points that their colleagues are urgmg. f=or 
exaiti|>lt>, Ai might fomlly imagine that he can meet tte requirements of hy ^Hing a sentertce here, a footnote 
then?, iifuf a fwragr f)h somewtien? else. In reality, the knowfeitge that has been pre supposed in unit 5 might, if 
taken MH^iously call for a railical restructuring of units 1 4. By the time A, realises this, it is too lattJ to <lo anything 
alKHit 4t. And iitiit 5 will therefore he tkiildtng m skimpy ami fnat!et}uate k^indatsons. 

Insteaii of using outline drafts as a first basis fw communicating intentions ami r^utrements, it m^t 
thot«|ht that ktn*?r flrafts (e.g- of 5,(MH> worth or so) frnuM be more apf^oprtate. This is not the case. Longish 
<fr4if ts, when produc«Kl in a hurry, tend to be discursive tours around the kinds of topics that the writer is hoping 
to <>)ver; |ust as soon as he gels down to the business of writing seriously. When the serious writing te^ns, the 
ina<lequ^ies of each |>erson's first thou^ts will become a«jarent. Certain topics will be droppmf, otheis wilt be 
intr<Klu<jef1, and just a few may be selecteil for treatment in greater depth. Alterations may also be matte to improve 
the structure amf format of the initial rou#i drafts. It follovus that the drafts are likely to undergo considerable 
cham^ and. in consequei>ce, do not de^rve to be taken too seriously. 

If long drafts are firoduccd, there is in fact a very real dan^r tf^t chaos will ensue. Accusations wilt be made 
that certain colleagues are planning to too muc^i material into their units. Predictions will tie irode that 
styles w<M clash, and that units will lack continuity. Sug^stions will be made that the whole course needs to be 
"re-thought", and that the ord&f of presentation of certain units should be cf^nged. A deefw?r f^oblem arises tt 
some aca<k*mic members of staff fail to umferstand each other's subject matter. If A3 and A^ come from different 
disciplines, A3 n^ay do his best to make helphil suggestions to Aj - but his comments may nevertheless strike A2 
as t>eir»g iiaive aiul irrelevant. 

In general, the ^noduction of hurriedly written long draftt is not a good idea. It is irksome to have to read so 
much material es>H*cially if everyone believes (a> that it is going to be radically chan^, and (b) that it fails to do 
justici? to tht» fX)tential abihties of the writers. It can also provoke endless worries and arguments. And any attempt 
to alleviate sudi worries <L».q. hy the fKoduction of hurriedly written amendments) may well have the opposite 
effect of haf ijt»nin(j rf.'sistance arKl/or generating yet another cro\} of anxieties. In this respect, the lireathless 
miyity of writing, circulatmq, and rewriting <loes not lead to a rapid convergence of opinion. The procedure is not 
selfcorrectinq. 

Constraints of Time and IVtoney 

It might bv thought that ad the mam difficulties and stresses and strains could be avoided by mea.is of appro- 
priate forward planning techniques. As might be expected, careful planning can gHsatfy f^ilitate the produttion of 
corrt*s|>undence and other materials. But it is important to realise that the nature of the operation, and the time 
f?ressure urtdi»r which evtnyone is working, are such that planning cannot possibly solve for even foresee) alt the 
fKOblcms thtit can arise. 

Sup^K)st% for tixample, that the academic members of staff (tecide to strengthen their course by the adilition 
of prescrir)«l liarkgr ouikI rcaiUnq materials. The most economic way of getting such materials to the stuctents is to 
arrange to have them published, on the open market, as a 'Rpader' in paperback form. However, it takes time to 
ni^gotiate with publishers. And it takes time to select the contents of the f^tatter, and to secure copyri<|ht clearances, 
and to aqf(K? eduorial chant]^»s with the original author and publisher, and to insert editunal comrnents and sumrr>arie$. 
ai»d to ^ftauqv for the whole volume to be set up, proof rear!, printed and distributed. The entire ventu*^^ is impossible, 
in fact, unlf'ss arf.inijt'mfnts to pnt>lish are put in hand m thf» very first quarter of the writing year. It follows that the 
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academic manbers of staff cannot hd¥e their Reatter unless tt^y are prerared to finalise its contents well hef<Ke 
most of their corresponder^ units haw actually fc«n vwrimn. Tfiis is a frustrating si* oition vvhteh no amount 
of forvyaird planmng can satisfdttorily r^ive. In the course of writii^ a unit, it often becomes clear tfiat the 
ancltlary r^Kiing materiah which vm dnnen in all ^lod faith at the course planning stage, is not after ail i most 
suitable <di<4ra* But there is no pc^ibility of sut^itutiitg a better selection because, oy the time the mistake has 
b^n discovered, plans for the Reader are irreversibly advar^ced. 

In addit'on to having to work umler acute time pressure^ tfte C^en University is also labouring under severe , 
economic constraints. This is a further source of frustration. If money were less of a yrobtem, ^^cademic miwbers 
of staff could afford, for example, to drop the idea of hurrk»diy concoctiim a one-volume Reader. They cuuld 
arrange, instead, for the last-minute imrtusion (in the student's c»rrespondence4>ad<age) of carefully selecttni off 
prints. Such an arrangement would tlwm more time to select the best possible back^ound reading m«nerwl 
But the cast would be very much hi^er. tecause of the financial restrictions u^idef which the University is opera 
ting, acactemic members of staff are obli^ (at risk to their nrofessionat reputations) to dash off a one- 
volume Reader which, for the reasons gj^^^ above, might well prove to be far from optimal. 

Even rH}W,we have touched on only a few of the many background pressures and ritatiurts that arademic 
members of staff encounter from day to day. Moreover, our discussion has been grossly oversimplified. In 
practi^, academics do not normally agree to write just 4 stKXressive units. Instead, they want tu be involved in 
the eiMite course — writing one or two units here, and a few more there. Quite often, they want to write mort^ 
than they have time to write - and each may feel that his ration of the aiurse is too small to cnnWe him to do .1 
worthwhile job. 

Another fact of life is that different ^::3demics have different working methods, and different views of ttn^ 
educational enterfxise in v^ich they are involved. One member of staff might be enthusiastic about computer 
marked <»>jective tests, whereas anotl^r mi|^t dei^ore them. One academic might prefer to write in a fnendlv 
and egalitarian manner, whereas others might adopt an impersonal approach. Tfvere an? literally dozens of ways 
in which differences of appro^ and opinion can arise. If tfte differences happen to reflect strong ur\6er^ /tng ccvn 
victions, then the possibility of open conflict is never far away. 

Oganisational Aids to Efficiency 

The foregoing acrount ts prc^bly a minimal statenrwnt of the main difficulties and pitfalls associated wrth 
the production of OU Murse n^terials. There are many additional difficulties of a more specialised and lt?ss 
obtrusive kind, some of which are described in a series of papers appearing in the British Journal of Educational 
Technology. See Lewis (1971, 1972). An important point to bear in mind is that almost every decision that fs 
made can have hidden and unexpected implications. Because of the constraints under which the OU is operstiiig. 
there is neither the time nor tfie money to irdulge in trial and error experimentation. If the right rtecisfons 
not made at the right times, production plans may be held up and the quality of the course materials wilt su^f^r 

One way of easing the burden is to employ a variety of judiciously-chosen assistants. It is helpful, for 
example, to have assistants who can enga^ in literature searches - e.g. to locate back^ound readmg matiTu/l'.. 
and suitable illustrations for proposed correspondence units. Tht?re is also u need for fH?ople who can handJ^ 
copynght problems, and who can help to «>-ordinate the diverse activities m which different members mI tjff 
likely to be engaged at any given time. In addition, there is a need for more speridlivtrd assist rioce in thr f^inri of 
editors and designers and artists - people who proof read, and advise on problems of format and layout, diid 
created special-purpose illustrations, and so on. Finally, there is an all-^rvasive neytl for professional adv^c*- a 1 ifu^ 
overall strategy and tactics of the whole operation. Within its limits resources, the OU has m fact secure<l a nnKtiM 
number of assistants and advisers of the kirKi required. It has also establisherl its own Institute of Edurat?on<jl 
TechrKilogy to advise on the design and evaluation of the teaching materials that are pr odijceti, and to mdkv 
recommendations that will help the University to become a rapidly so!f-improvinn sy^tt^m. 
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What eHe can be ttone to fdoiiitate the ptanning and f>roduct»on prcK:^? Well, thef e are some rather obvious 
l)ff»Ctnttiondrv tneasures that production teams can take. For examf^e, most teams find it convenient to organise 
thf»mst'lves, at an oarly staye, into str^atl warkirfg grmif^ of 3-4 memb^^s. Within each working group, one member 
h«i% i»inui V res|3omibi)itv foi writing a particular set of correspondence materiais lor for preparing a particular 
Mii t>roadtast materialsK atnl the other memiwrs have mainly an advisory or ^tc^iitrg lule. As a fin-ttw safe- 
guard, tht* wof king grou^K overlap in tt^ir membership - so that everyone can fumrtion as the main author in 
soint? grtiu|fs, ami ;k an advi^ or observer in other grou|». It follows that each academic can rerrtain in close con- 
tact with those coHoagues Ohose writnv] adi^ent units, for exan^ple) whom he is most Hkcly to have to accommo- 
date. In this way, inamtpatibiiities of approach have a good cham^ oi being piekeci up at an aariy stage. 

Tbi»re are, however, quite severe limits to what this kind of arrangement can achieve. It is easier to detect 
di .idfities of approach than to decide wrfiat ^ufd be itane about them. {Imagine, for example, the difficulties 
that can arise if three scientists, in overtapf^ng working groups, all stprt to write in different ways about "theories 
aiul models" t The setting up of overl^qiTping 9^ouf» also tends to produce only kicalis^, rather than extended^ 
cufitiMuity fhere is still 3 dancpr of conceptual drift <»:curing from one sequence of units to the next. To secure 
stunn suit of maiLio-control over the wholf> 32 units, it helps to have an overall diairman and art>fter who is a 
qefiet dhst in the sense of being able to span the various dfsci|;^ines of his collea^s - and who also has the 
knowUMiq4^ ami vision and charisma to keep evtsi yuite moving in the same direction. Needless to say, this is easier 
said than don£^. 

Some Recent Developments 

In recognition of the very real difficulties that arise w*ien te^ns of a dozen or nrore experts coltectively try 
to prcKiuce a full course of up to 9 months duration, several attempts have J:K?en made in recent rm>nths to simplify 
matters. O^ie |)opular strategy has been to abandor the quest for cto^-knit overall integration, and to deviate 
couf ses into several self contained 'T>l<KJks" of 4-8 units each. This enables everyone to work in much smaller 
qfOi^is, and to worry rr\u&i less atout what tlwir colleagues are producing in iheir blocks, A similar ploy consists 
in pr miucing %-courses and 1/3*{X)urses (and even one-sixth cxiurs^) of not more than 18 or 12 {or 6) weeks' 
duration each. In addition, the original aim of ^nerating 36 weeks' work {in respect of each full course) has been 
widtMy modified to a less gruelling IK)- 32 weeks' work. This reduces the work load on the course teams, and gives 
the student a vefy welccyne week off, here and there, in whicn to catch up or revise or just rest. 

St?vt?ral other si..ipltfying strategies exist. There is, fnr rnample, an increasing tendency to rely more heavily 
on alreac^v existi'H] texts - mstea<l of trying always to create special f>ew materials from scratch. In view of the 
clash of viewpoints and styles that can arise when different ^ademics contribute to a common course, some 
acadi»mics have also agreed to hand over the main writing chores to just one or two Hk^-mlnded colleagues. This 
moar^s that, for one course, just one or two ^demies r. take respcnsifciJity for the bulk of writing, the remain- 
ing academics acting as their assistants throu^out. In Sv^ iequent courses, the roles wilt be reversed. It is too early 
to pass iudgement on the efficacy of these and other strategies. The devision of courses into loosely-related 
"hlorks", for instance, might well disturb the insecure student who wants well-integrated materials. But it miglik 
well appeal to the kind of student who wants to think for himself. The limiting case would occur if every course 
Xi\im duthcK dec(ded to "do his own thing" in polite disregard of the conlnbutions of his colleagues. We would 
then hdvf thr sort of teaching that goes on in almost every conventional university. 

Activity Networks ami Schedules 

In order to plan and produce a complete course, it turns out that several hundred quite complicated decisions 
h»ivr to he routinely taken and acted upon. As additional aids to efficient action, special charts are needed to enu- 
mi«r^to, as ('xhaustively aS possible, what thiwe decisions are, and who should take them, and why and when, etc, 
Supporrinci tables arp also required to pomt up the major implications (financial organisational, psychological, etc.) 
ttni! pjftjculaf (kTJSions aft' likely to havi^ both in th'' short term and in the lonc^r term. 
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There are numerous v^ys of cteprcttng the production profess in dia^ammattc form. Eadi diagram that is 
drawn inevitafaly highti^ts some ft?dtures of course production at the expense of others. And %o single dia^am 
can possfbiy do fustice to th^ full complexity arxl variability of the production proct^s as a whole. 

Dt&gram 6 has a certain hi^orical interest. It rci^esents the wty first attenrpt {in 19^) to give a brief 
diaracteri&ation of overall 0roductk)n process. The dia^am partitiofis tfie pro<^ into 3 st^p^. Sta^ 1 is con- 
cerned with the progressive clarif icatii^ of a>urse team intmtf<^ - startin9 v^ith va^ k^eas about the natin^ and 
ethos of the coio^sa as a whc!e, and ttnishing with ctear-cut id^s about the proposed content and rationale of 
every compon^t of the course?. Sta^ 2 is concerned with tite progressive impierr^ntation of course team intentiom. 
And Sta^ 3 is OTfKern^ with the purely technk:^ (but difficult) business of final prwk^thfk 

Diagr;»m 7 is a tater characterisation. The terminology is different. There is a stror^r hint of the need for 
certain activities to go on cowurrentiy. And there ts a more explicit recognition of the need to re-cycle (e.g. to 
revise aims and ob^cttves and initial drafts of units, in the light of on^ping experien^). Dta^^ms 6-7 are« however, 
essentidiiy t;oniH>dtibie, Itnderlying each dia^am is the reco^ttion that effectiw course f^'oduction calls fw the 
kind of rational planning which enables initially hazy ideas to firmed up and tried out (e.g. in the form of 
tentative drafts of correspondence malBridisJ and subi&cted to critical discu^ion and reviser! again, so that in the 
space of not more than a year ~ a polished and weHnnte^d^H course can cmer^. 

Diagrams 6 and 7 have a useful orientatfng function. They capture the most important and rrK>st commonly 
recurring features of course production, and thus provide the outsider with a helpful overview. But they were 
never intended to be used .iS practical ^id^ to action - fcK the simple reason that they do not rontain nearly 
enot^h chtail. In order to see this, it is worth looking at diagram 8, whidi gives a fairly comfK^hertsive list of the 
"products'' which a)urse teams may be called uiwn to generate. 

As diagram 8 shov^, the main products — the primary and sea)ndary coi^se matariats, home experinr^entat 
kits, homework assl^ments, radio and television programmes, and the like — are devised for the direct benefit of 
the student, in addition, a considet able amount of explanatcx^y material has to be prepare m>d ctespatched to other 
Quarters — e.r *o regional staff, study center staff, a>rrespondence tutors, summer school staff, examiiwrs ard 
moderators, data iM-ocessing staff, aiKl the publicity department. It r^uld a b^g mistake to underestimate either 
the difficulty cm- the importance of producing this ancillary {$upfX)rting) material. Time expanded on providing 
explantory notes for study centre tutors is time well spent. So a!so is the time devoted to thinkirtg about summer 
schools^ and to drawing up sensitive marking sdtemes for hcxnework assignments. Any production scheme that 
failed to stress the need for such items wouki seriously def^ive as a practical gutde to actk>n. 

To comprehefKi course prcxiuction at this level of detail, some rather lar^ charts are required. One such 
chart — de^ibed an auUvity network because it depicts a lar^ interlockirtg networ' of essential course team 
activities - covers 8 pages of a paper by the f^esent writer {Lewis, 1971b). It is unfortunately too large to be 
reproduced in :his chapter. By way of compromise, diagram 9 acwrdingly shows just on small segment of e rather 
simpler network that is actt jtly being us&i by one OU coufse team. 

Networks of the kind illustrated in diagram 9 can be embellished in varbus wavs. For example, they can be 
annotated to show the approximate time (in days or half-days) ih^t each activity is likely to take on average. They 
^can also be annotated to mdtcate vn^o ought to have primary responsibility etc. for each of the activities shown. 
However, it is important to note that each course h^ its own distinctive dnaractenstics and requirements, so no 
retwork can ever serve as anything more than an overall ^neral guide. In particular, time estimates can l>e 
notoriously misleading. Certain activiti^ can take anytr.iry from a few hours to a few weeks, depending on how 
much time and effort the course team members care to l^wt^h upon them. See tewis (1971 c). 

What, then, is the value of such networks. Well, their main value is to act as memory joggers - to draw 
anentjon to what needs to be done and to sug^t one or two ^kkI ways of doing it. In addition, of course, it is 
necessary to have a fairly firm set of schedufes. In the last analysis, the clock cannot be ignored. So it is necessary 
for course teams to realise that extra time spent on one activity means less time spent on another. 
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Each course warn accordins^y operates a nunr^ of backward-runmr^ schedules of the kind shown in 
diagrams 10 dini 11. The«j schedules are backwanj-runnii^ in the seme that absolute deaditms are determined 
(e.g, the latest time for settling copyright problems, or handing over p^aphics, or completing final draft of corres- 
pondence text), and the work ts then spread as evenly as possible ov^ the weeks leading 14) to the cteadiines in 
question. This typically leads to the production of a large matrix in wh^ a separate column is aligned to eiN:h 
key ^rtivity, and a separate row is s^stgned to es^h week of the year. In each mH of the matrix it is possible to 
record the stage that a ci^ain activity should have reacheii by a certain w^k. It Is also possible to record the 
name of the p^on($) responsible for handling that stage of lYm activity. 

As a further safeguard, the Vice-Oianc^lor's office ot:^ates ager^al nK»nitoring system over course 
production as a whole. In this way, it is possible to fWdict the kinds of demarals that are likely to be made on 
the Media Production Department and on other parts of ttw Open Uni^rsity's system, and to cxxwtlinate the 
flow of material from the individi^l course teams through the production and desfwtch system, and on to the 
student. Arrangements of the kind just descrited are fay no n^ns qotimai, but they frave so far ef^bled the OU 
to avoid serious crisis. 

Furtf^r Reading 

A continuing series of papm, by the presem writer, is appearing in the British Journal of Educational 
Technology. The first 5 in the series are: 

t. Lewis, B. N., Course Production at the Ojcwn University t: some basic (^^oblcms. BrJ.ed. Terh. Vol. 2, 
No. 1, 1971a. 

2. Lewis, 1. N., Course Production at the Open University II: activities and activity networks. Br.J.ed. Tech. 
Vol. 2, No. 2, 1971 B, 

3. Lewis, B.N., Course Production at the Open Uni\^rsity III: planning and scheduling. BrJ.ed.Tech. 
Vol.2, No, 3, 1971c. 

4. Lewis, 8.N. Course Production at the Open University IV: the problem of assessment. BrJ.ed.Tech. 
Vol, 3, No 2, 1972, 

5. Lewis, B.N., Educational Techf>ology at the Open University: Indices of quality, Br.J.ed. Tech. (In press), 
1973. 



216 



\ 
\ 

\ 



BEST CuPX /^ial^DLE 



00 



ERIC 



1 



I 

3 



s 



1 
I 

5 



2-^ 



U1 



CO 



9-> 



U1 



CD 



U1 



''Recording" 

Programme 
recorded ?.t 

"1st copy" 
Copy Auatlable 
at OU for 
viewing 



t^ision' 



"2nd copy" 
Master tape 
delivered to 
Technicolor 



"Sample" 
Answer print 
approved by 
AVAA. 



"Delivery" 
Cassettes Delivered 
to Regional Offices 



JUDOiNG PRODUCTIVITY POTENTIAL OF AN INSTRUCTIONAL SYSTEM: 
A REACTION TO PAPERS BY BALDWIN AND LEWIS 



Stephen Yelon 
Prof«^r of Educat'tpnat Psychology 
Michi^n State University 

An eiiticator often has to judge the pfodi^tvity potential of an instructional system, a i^oposed instruction- 
al system or one m practice He can do so by checking to see if a project director uses procedures ^ich are likely 
tu create an instructional dcsiqn wrhich in turn is likely to result in ino-eased learning, quicker learning, less exfwn- 
sive learning arKi more acceptable learning. 

A project director can find procedures leading to pffodiKrtiw desi^i by abstracting from the prot^ures used 
to form proven productive instructicMial systems. He can comoine what he f IihI* in exceptional systems with what 
he knows about mocfels aaj theories of instruction. The ^rpose of this paper is to give a diedc list of pr«»dures 
to project directors. 

There are five basic promiures that must be included in a potentially productive instructional system. A 
proposal should include procedures relating to H ) The measurement of productivity. (2) The derivation of goats, 
(3) An account of resources and limitations, (4) An application of social science, and (5) An evaluation for 
improvement. 

(Vtea«uremeni Pf oceoures 

Let us look at each procedure; the first procedure results in the creation and use of measures. One asks, 
"Does the proposal include plans to have an ob^fCtive agency measure many aspects of the productivity?" 

To bp more productive, a project director must know how well his system is doing in all possible respects* 
To measure many aspects of prodijctivity one must go beyond using credit hours or courses listed. Productivity 
may mean any of the following things: 

More students achieve important objectives. 

More students apply what they hav'e learned in the real world. 

More students remembti what they have been taught for a lon^r period of time. 

More objectives have been achieved in a given time. 

The achievement takes less time for planning and for learning. 

The achievement takes less money. 

The achievement takes less work. 

Students, faculty, and people in the community are more satisfied with the results. 
More students riemarKl the course work. 

Measuring varied aspects of productivity is imperative in an opisn setting where a project director is never sure 
what resiJits h^* is iioiruj toqet bncau^' of the expt^rimenlal ti^chniques often used. 
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An obieci.w agency should be used to evaluate productivity because it is often di«icult for one to see 
own tf rws beyond his experience. The objective agency should be called in to measure the productivity of ttie 
system both as it is being developed arwl when a prototype is complete. 

- Although Baldwii; did measure some aspects of product-vity he certainly did not measure all and left out 
some important productiv.ty measures. He talked of many offerings for less money, added enrollment, lesi wor* 
for more course offerings, f .^i,»g of schedules and making learning more acceptable to students. He did not how- 
ewer, mention students applying what they had teamed in the world, or adiieving more objectives. He did not 
mtjntion anything about an objective agency doing the measurement. ' 

Lewis, spoke of 900 graduates ami 40.(00 students enrolled, a low dropout rate, and other measures of 
acceptability to the students and to the community, but he did not talk about productivity in terms of student 
learning. 

Goal Oeri^dtion Procedures 

The second imponant procedure to create pr cxJucttve instmctional d^ign is the derision of meaningful 
goals. One asks, "Does the proposal include procedures to derive meaningful goalsr To be productive a project 
r^iust meet stuiient need$. Therefore, a project director must assess student needs and then set goats which meet 
those needs. 

The assessment of a community's status to determine prerequisite adiievement, attitude, belief, skill and 
mterest is crucial m an open system where students have little direct contact with staff and where students are 
quite different than those attending traditional colie^ and universities. Goals must be stated to communicate 
instruct.onal intent to students and staff. The goals must be communicable, comprehensive and reality based. 

By creatmg goals based on audience assessment, a projea director can make learning mud> more meaningful 
to the students; and therefore, increase the completion rate, increase student satisfaction, increase enrollment 
and increase the number of objectives that students will learn because he has accounted for their pre-requisites. 
When a project director states goals clearly, students may select courses more accurately and are likely to achieve 
a certain amount of satisfaction by recognizing the results of their labors. The faculty are likely to plan and 
implement their teaching with greater efficiency and are likely to be satisfied when seeing concrete results. By 
eliminating irrelevant objectives and adding important ones, a better use will be made of project money. 

Baldwin alluded to some assessment of the needs of older engineers, but it seems that this assessment did 
not come before setting course goals. Course goals wrere not stated in the paper. Lewis reported on assessing 
student needs and stated that staff continue to survey student needs, interests, and progress. Course writers 
assume students do not have any prerequisites for a given course and let the students take exemptions to find 
their proper placement. Practical goals are used by staff for writing course units. 

Procedures for Accounting for Resources 

The third procedure used to create productive instructional design call for making the best use of the 
resources and l.m.tdtions present in the system. One asks, "Does the proposal include procedures which will 
creato a design to take mto account the resources such as time, media, staH, students, community, money, 
spacif. and course content. The general strategy a project director should follow is to maximize the usefulness of 
the rt'Soiirci'S and work around or reduce the limitations. 

In jn open system a great deal of time and money mav be spent on expensive media to reach students in 
remote areas. This time, money and media are three resourras that must be used vciy well. 
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A $.^0)t!€t director can use time wilt by carefully ptanntrig production strategies. He OHild ^nerate a proto- 
type early to teit his ptonning pr oceiluits, his teaching procedures and his testing prooeckires. He can use materials 
similar to those he will generate. He antUi crealf a prt^am with the barest essentials of content then, as 
empii ical evidence is colltycied, he i chjUI *iiW ntfi:t*ssitii!S to the program. He could plan to keep all of the units in 
draft form except for the first orK!. The lessons learned from testing the first unit rould be applied to the ^osnd, 
tfien the serarid unit CiMjId be tt!Sted. He oiuld filan for his planning; ho couM assign responsibilities and set dea<1- 
lines, In practice he could ti^st stiidents and materials frequently so that any remedies would be smalt enough to 
tie handled t^uickly and int^xpensfvely. He could create instructioodl products which are self-contained, t€nit€^« 
and re-usable. 

Baldwif), savtKl time bv protlucing instriicfionai pfiKlucts, and ^aved tf^ instructors from repeating Hiin^ 
that they had already done. To s<^ve time tht?*;e ivt^lucf; were iised dutmg working hours and for remedial work. 
No other strategics to save tinre were mentioned. Lewis, saved time in n^ny ways, he provided st i^errt exemptions 
of courses, by having fiK:uity work ijrpups analyze rou^ outlir^ drafts, by using objectives ^d conceptual dia^ 
grams to get the students to learn more quickly, hy trying to work around the problem of seqtiencing of instruc- 
tfon, by using activity schedules to set tlearllines and assign respoi>sibihties, by using existing texts and by ^tting 
the system out and ready for te^^tinrj. 

Fred Allen, the noted rjJin corned; jn, :>'r;CJ U3*d "The reascti they call radio a medium is that nothing on it 
is ever well done." All project directors are tol<1 to uf^e media appropriately; however, there are very few people 
who know what the approprut** use might be hyr different forms of nvedia. Baldwin u^ d television to extend 
place; whether he usecJ teU?viSi4)n apf>fopnately, we cannot tell. But baldwin seems to be wedded to vkleo-taP0« 
and may have eliminat^.Hj i ;her kinds of meilid cfevelopmeiit which nwy be o^nsiderably less expensiw and 
perhaps, in some cases morr appK>^>rjate. Lewis uses all forms of media, each with n particular rationale. 

In an opi^n system where the maxinium amount of students have to be reached, the staff must be used well. 
To do so an administrator mu^t employ strategi&s such as division of latior, team work, and work sd^dutes 
includmq deadlines and rcsponsibilitu's. He must work at increasing the staff's motivation to do a good job by 
using ^>aftfOi|fattve manaqement, by pfoviclmg concrete and direct rewards to the faculty, by employing com- 
munications ?;ystf?m5 so that the staH i;ar> work well togetlHjr, by tr jining faculty in learning system design*, in the 
psychuloyy of learning, in the exact Miiture of then system, m tht? ability to work together as a team and in the 
abiitty to use resources. He may (*vt^n qo so tar as t<» Ik? sure that The values of the stdff members coincide with the 
goals of the system. 

In all of tht^se ways a pf ujt'cl diri^ctor can increase the satisfaction of the staff by »jivinq them control. He can 
increase the prooabihty of le?^f ninq by f<fic1iTU| good suggestions throu^ participative managemem. 

Baldwin, used staff to create the video tapes and otxasionafly to visit the students. Lewis, used staff by 
employing methods of division of lat>or and tedrnwcrk, by setting up j system for communicating criticism to each 
other, by dividnv] qroups mto writers fit u1 oUstfrvers, by choosing p4?ople of similar views to wr»te rather than to 
fight \i out, and by m'suh} tutois to (ic ttie \nb of human contact and immedi:ite feedfiack. 

Ui an open university systt m ^.taff cannot possibly iio dtl of the teaching and cover all cf the remote areas in 
which r»ttidtMfts'' will l>e piaccd, thus, th*.* slucients most neijlected i /voufu: m any instructional system, must be 
used well. Tht;y muit tauqht to tutor imc^j other and to tH^netit Irom th«* system. Baldwin, did not mention 
anythincj rcldted to the use ot students Un peer teachif^ij. Lewis, took the students if »to accrvunt by easing the pace 
of assignments and by plannmq to teach students to teach Hdch oth(*r. 



To use money vv«H, an accurate as^ssn^t of the costs must be made. A project director must learn how to 
estimate costs accurately and learn how to amortize the costs over time. He must take into account the irr^iort of 
the i|oa» for whtch he is sliding money, he nmst ask how much money is it worth to reacJi a given ^al. Few 
mstnictiOfTal ifevelopers, including Balilwifi and Lewis, speak of the ciuestion of money related to ^als. Baldwin, 
however, uses money weJI by making an accurate assei^ment and being able to amortise the costs of course iwirk 
over time. This ability contes from years of exf^ience of prorfucmg video tapes, Lewis uses money welt by not 
spending any more than luHl«^teri, by n?ctuiring staff to put in more tin^, and by not wasting money by f^oducing 
special books m6 extra, costly, unnecessary media. 

Procedures for Applying Social Science 

The fourth procedure for creating i^oductii^ instructional design is tfte application of fHlnctples. One asks, 
"Does the jiroposal inctucte prot^tiures for applying social science principles, those that inducte the psychology of 
learning^ motivation, and attention, and the principles of sociology aitd anthropology?^ If a f^oject is to be 
productivt*, new materials must work the first time to reduce additional cost and time for revision. A project 
tfirector must use what he knov^s from socicrfogy to assess and take into account the characteristics of certain 
qrnu^is. In open systems this is inHKHrtant, it is also imiXHrtant to apply general prindptes of learning, mottvatkin, 
dttentiun, communication and perception. 

Bdldwin descTiH<»s practice, assignments, and visits, but I cannot be sure in reading his paper, whetha^ there 
vvd^ «i cofictnted effort to introduce psydioJofjical principles into the creatk)n of the systeni. Lewis, hovi^ver, 
provfiles clear evidem^ of the use of psychological principles dealing with practice, feedtiack, mferaction, 
fxiji^ressive testing, repetition, use of objectives, use of knc^iedge structure, mni so on. 

Formative Evsrfuation Procedures 

The hfth procedure for creating prodiM:tive instructicmal design is formati^ evaluation/ asks, "Ooes 
the pioposal include procedures for self -improvement throy#i data collection?" To be productive a project director 
has to be committed to improvement- To improve a project, a director must collect evidence to find areas of 
Success and areas of failure* Even if all other procedures are put into effect errors will tic made because of the 
presence of new audiences and new techniMues. fcvidence must be collected in ordw to eliminate those errors, 
Baldwiri <lescnbes some research done to improve one of his pet projects* Lewis states that the formative 
evaluation process is built in students, tutors ami survey researchers are always collecting data to find out where 
the system can be improved; tests are analyzed for this purpose as yv^ll. 

The presence ;:f these five procedures do not guarantee productivity because there are many problem^; and 
many issues related to each procedure. In i^neral however, if these f iwe procedures are included to plan an 
instructional system, the chances are that the system would be productive. I tH?lieve that if in the long run the 
instructional systems described by Baldwin and Lewis are productive it is becaiirr they planned using procedures 
likf» those described to design their systems. 



'Ac yPT ^iHhnufn^ tr» not w/nll forr^MiM^o^^ tujT t h^tv*^M'o<nplf?tCf I rit^U mijni,srrtf>t ot .3 text on formative evaluation, 
v^hich will (oon he availat)lc. 
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A REACTION TO '^CHtCAGO^S TV (ALLEGE: 
A TELEVISION BASED OPEN LEARNING MODEL'' 

G. Klare 
Ohio Unfv^ity 

The su(X^s of Chicago's TV CoUege, as described by Dean Zigerell, is* most impressive. Lt?: me comment 
briefly on the maior indices of success now and come t^cfc to them a0din idler. 

1. Progratn duration. The TV CoO*»gc has been m existeiHie for 17 years-many times longer than most 
instructional television, or ITV, efforts. 

2. EnroHnwnt The number of students re^ed during the College's existence is over 150,000, with mut^t 
than half taking courses for tredit. This makes the TV College by itself as large as a state university in 
terms of students taught* 

3. Satisfactioa. The satisfaction registered concerning the T^/ instruction compares favorat?iy to that for 
conventional college courses, in fact ex^eds it in some r^ards. This is surprising in view of the less 
positive results of most early ITV r^arch covering the same kind of instruction. 

fi 

4. Cost, The cost per student-credit-hour is very dose to that for conventional courses, in fact, slightly 
less. This is in many ways the most impressive of alt indices. 

Rather than say more about the indices of success now, it would be more profitable, ! think, to look instea4i 
at the probable reasons for success. The most impressive area of suMess has been with the largest segment of the 
audience—the homeviewers. who make up 75% to %Q% of the a^it enrollment at the present time. Let me, there- 
fore, concentrate on the probable reasons for success with the homevlewing audience. 

The logical place to look first is at what the literature in ITV says, to see if the Chica^ TV College's exwr nee 
bears it out. Two related points are frequently cit^ in the literature. 

1, Marm& of presentation. This is frequently given as aitical to viewer acceptance of ITV. 

2. Use of s^yecialists. This is often described as n^c^ssary for effective use of television as a mndium U)\ 
instruction. 

Did the Chicago TV College follow these rlicta? Let's look, beginning wilh manriLT of presentation of prni|f,j/f>:,. 

It seems a!most axiomatic now in ITV that one does not pres<?nt a so-called "Talking-head" or "talking facti" 
unless, perhaps, they happen to belong to someone like Bill Cosby. The message seems to be, that a r^^gular n^stnirtor 
should not stand in front of the camera as he would in talking to his class, at It-a^t not for us long *j (UMfter tjf 
semester. As we found in our large introductory psychology classes of 2,000 some years atjo, stud^'nts seem t(5 
prefer a hvf? human beimj f;vf?n if tln^y cyn only set; him with fnMfl (jU^si'^. 
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Yet, Oiir^go's TV Caftei^ iioes not seem to have hiwn lao concerned with this matter of manner of pr^nta- 
tfon. As Dean Zigerell says, the TV courses are basically "... a visually er^anced, carefully ret^ars^ Gl^room 
performance/' He also says that the Ck>liege does not use "^fast -moving, hi^-visuai pf<Kiuction tediniques.'^ 

Next, let's look at the m*j?ter of use of H>^iaiists. Here, again, the TV Coile^ does mx seem to have foliowed 
the "game plan". As Dean Zigereli says, "instructional teams have seldom b«2n employed/' and "TV College lacks 
. . . the resources of money and personnel for thorcnj^ly 'n^iated' ir^uction." 

When then, has Chtcago^s TV CoMe^ been so succ^ful? Dean Zig^ll has I think, (^ovi<ted ^e cluis in his 
address. But I would tike to emphasize them, so that it is ixmible to suggest ways for others to profit from his 
experiences. He says, at one (K>int, that the Chicago metropolitan area contains a '"virtuaRy hiexhsustibie 
audience of mature, able, highly motivated students eager to enroll as aedit students in oolles^ level televi^on 

courses. 

I'm sure Dean Zigereli chose his adiectiv^ cmefuHy; 

a. "highly motivated/' 

b. "able/' 

c. "mature/' and as he added later in his pai:^, 

d. "largely female/' 

Ami just m case some additional characteristics should needed, VW leave a place for "Other/' 

As an instructor, ! c^n't help remarking, that hi^ly motivated, able, mati^ women students are likdy to 
learn under 3ny teeing method or any instqiACtional ixmditlons, no matter how good w poor. As most of us 
kik)w, it would be pretty difficult to keep such students from learning. App^H^ently tfie only things that did, in the 
case of the Chicago TV CoHecfe audience, was the lack of ao^^ibility to education ar^l/or tuition funds. 

Given such a situation, I think cx^-tain imp^icati<xis follow from the Chica^ TV Colly's experierKes when 
others— or even Dean Zigereli and his group--wish to serve different audiences with tlhe same success as his current 
homeviewing audience. I think it can be irstructtve to look at these implications as a help to others now, tnJt also 
to suggest soTie needed research as Chicago's own audience changes. Since Dean Zi^rell says that an attempt is 
being m3de to reach the disadvartta^d se^ents of Chicago's population, we might look at the implications with 
that kind of chanced audience in mind. And again, use the applicable indices of sucrass that I mentioned earlier. 

Implication One. The most interesting and important impticjitian is that the characteristics of presentation 
will probably need to be changed for a new audience, at least if enrollment is not to suffer. What kinds and 
degrees of changes will be needed? That is a question for research-or trial-and-error— of course. But some 
hypotheses are suggested by the TV College's experiences. Note that four likely candidates for chan^ presenta- 
tion are the amount offered of: 

a. fast-moving, high-visual production techniques; 

b. seif-sconng and programmed mater iais; 

c. face to-face and telephone conferences; and 
vocational orientation in courses. 

Again, i have added an "other" category to cover other possible characteristics not listed. Given subjects who 
are highly motivated, high in ability, highly mature and in large proportion women, only relatively small amounts 
of the four program characteristics are needed. But, given subjects who are, in some ca<nbination, less highly 
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motivated, less able, less matur« and/or are in smaller prc^ortion women, which of the ip»^Q^am diar^teristks wiH 
need to be ct^n^? Only time can tetl-and I hope it mil tell That is, I hope that ^ the prime audience for 
Chicago's TV Colle^ changes, an attempt will be made to relate the chan^ in people characteristics to those in 
program cha«'acteristtcs. 

Other program charactiw^istics not fisted may, of course, need to be changed also, \ woutd like to sugs^t one 
myself, in fact. I believe that the instructional material used, particularly the written material, will probably have 
to be made more "readable," i.e., easi^ to understand and more efficient to read. As motivation and ability go 
down, the need for improved readidritity goes up, if materials are to remain effective. ^For a brief discussion of 
readability and its effect upon reader l^havior, see tfie attached paper, "Readability and the Behavior of 
Readers".) 

In summary, keeping enrollmait up with a less easily taught audience will not be easy. Such a chan^ has 
other implications for the indices of suoress mentioned earlier. 

lmplicati€Hi Two, The second implication is that the satisfaction expressed by the stuilent viewers is likely to 
decrease. At least in the early stages, student comments are likely to move in the direction more typically found 
in evaluation of the type of presentation used in Chicago. This is likely to show up in something of a drop in the 
hi^ current numbers who: 

a. say they enK>y stutlying on TV, or who say they like their TV courses; 

b. say they feel they le irn as much from TV as from conventional classes, as is now the case; 

c. say they find their TV courses better organized and presented than conventional classes. 

It IS hard to say how grades in the TV wwrses wilt compare to those in conventional course*, since grading 
tends to be so subjective and thus subject to other pressures. It woutd seem ctesir able to comp^ve actual grades 
rather than, as in the study reported, to ask students how their grades comf^red. Human memory being what it is, 
1 usually find self -reports of grades somewhat hi^er than actual grades. 

Implication three. The cost of ITV instruction is likely to increase. It will be interesting to see whether the 
cost per student^rediT-hour can be held as low as for conventional instruction. This is not meant to be a aiticism 
of ITV: I hope it can be kept as low. What realty concerns me is whether Dean Zigerel) and his youp will be able to 
get extra money to vary their presentation if this is needed. I hope, especially, tliat some research money will be- 
come available to stuoy the characteristics of the new audience and to try various char^ in presentation in order 
to maintain the success currently being achieved. 

In summarizing, let me make several pointe. My comments w^e fK)t meant, as I've indicated, to detract from 
or disparage the success of the Chir;igo TV College, As Dean Zigerell has saki, an ITV presentation must be evaluated 
in terms of what it sets out to do and for whom. What I hope, is that it will be possible to document the ways the 
current approach must be chan^d to deal successfully with a changed audience. 

This will take research time and money. But it could be very worthwhile fur any other ITV programs in the 
planning^ Of even operational -stages. With the current growth of the open-learning and open-university models, 
thi«; rould help many states- and perhaps also some countries-to move faster along tho road to effectiveness. 
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READABILITY AND THE BEHAVIOR OF READERS 

G. R. Klare 
Ohio University 

One of the best ways to teft people what "readabiHty** is, I've found, i$ to start by burning them what it 
isn't. I happen to have her^ wt^t I think is a particulaily gtKKl-or had-exampte from the Fetter R^^f. 

The maximum price for a primary fish shipper sale of fresh fish or sea-food ^except shrimp, 
salmon or halibut) to a retailer or purveyor of meals where the sale is ne^tiated or matte at a 
branch warehouse as herein defined and where the fish or ^a*food is sold and delivered from 
the stock of a primary fish %h\f^fm wholesalers branch war^rouse which is remote from his main 
place of doing business, and at whidi ward>ouse the {^imary ^ifMser emptoys two or nwne full 
time employees who are stationed at and engaged in making sales and performing ^ vices solely 
for the primory fish shipper from wich warehouse, is the f^ki& listed in T^e O in 22 plus the 
allowance provided in 6 for a s^ce and ctelivery sale where sudi $^ie is made, plus ttte trans- 
portation alioi^nce in 9 plus the appropriate comainer altowmce in 21 * 

After reading this passage. 1 think you can see one of the reasons that price controls may be unpopular with 
those who have to apply tt>em. The passage illustrates one extreme of the most common dictionary definition of 
readability: relative ease of understanding or a>m^eher»ion due to the style of v^iting. This definition contains 
the two points rnost commonly agreed upon concerning readability: 

(a) the emphasis upon "understanding" or "comprehemion", and 

|b) the eraphasis upon "style in writing" rather than content, i.e., upon how somethinq Is said rather than 
upon what it is said- 

Those two points will come up again at different places in my discussion, which concerns the relationship of 
rcadabflity to reader behavior, with special attention to reader persistence. 

in order to establish a relationship, of course, a measure of the variat^es involved Is needed. Let me begin 
with readability. The most common measure of readability is the socalled "readability formula." Well over 100 
such formulas have been published (my count to date is 141 ); 33 were deyeto^mi by one research worker alone> 
m recent years. The large number is partly a reflection of the many different purposes and materials to which 
formula? may be applied. But is is a*so partly a reflection of the convenience of computers in maniputatirrg and 
analyzing the data needed for formula development and for formula application. At lf>ast 29 computer programs 
currently exist for applying readability formulas to larc^ bcKjies of data. 

I do not want to get deeply into the topic of readability formulas and their variety and diversity. That is 
another topic for another day. Instead^ let me simply provide a capsule description of formulas. Fortunately, this 
can be done quickly as follows. 

1 . The variables in readability formulas usually consist of cuunts of langua^, or style, elements which 
have been found to correlate highly with comprehension scores on a wide range of passa^s. 
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2. The vanahlHS aio usually chosen so that, m addition to thiPir predtctiveness, they are objective, quick, 
ami edsy to a$.i|3ty. 

3. Ttie most commonly used variables are some measure of W(Hd difficulty, such aa familiarity or length, 
jinl s<mu» ni<?jsufi! of sontt»r»ce difftcirfty, su<^ as complexity or letiqth. 

4. The vtinables are usually combined in a regression equation. 

5. The formula ts usually apptted to s^pl^ of written material, especially when the material is lengthy. 

6. The formula provides scores w^idi serve as an imlex of tli^ feadability, i.e., tfie reading difficulty, of 
the written materiaL Sometimes grade-l«vel sct^'es are usetJ, ami sometimes a different scale i. used. 
But the interpretatitm of the s<^res, at least, usually inv^ves yade leveb tiecause their meaning is easy 
to grasp and apply in an educatic^al situation. 

LvA im? show you a typical readability formula. It is the Rcsch "f^eading Easy" formula, with 
that is commonly tised to interpret the scwes it provides. j 

Table 1. intefpretatton Table For Fleidi Reading Ease Scores 

/ 

Formula R.E. 206.84 Bbi^l - 1.015$/ ^ 



R. £. 
SCORE 


4 

DESCRIPT ION 
or STYI-E 


TYPICAL 
SYLLABLE 
LENGTH 
{too WORDS) 


TYPICAL 
SENTENCE 
LENGTH 


TYPICAL 
MAGAZINE 


POTENTIAL 
AUDIENCE 
SCHOOL GRADE 
COMPLETED 


030 


Very difficult 


192 or nwre 


j 

29 or more 


Scientific 


College 


30-50 


Difficult 


167 


25/ 


Academic 


H.S, or some 
College 


50 60 


Fairly difficult 


155 




Quality 


Some H.S. 


60-70 


Staniiani 


147 


17 


Dic^sts 


7-8 Grade 


7080 


Fairly Easy 


139 




Slicf^ fiction 


6 Grade 


80-90 


Easy 


131 


V 


Pulp-fiction 


5 Grade 


90-100 


Very Easy 


123 or less 


6 or less 


Comics 


4 Grade 



4- 



The formula, R E. - 2-6.84 ^ ,85iv/ - 1.015s/, is at the top, R.E, = "Readimj Ease," which usually varies 
from 0 <which is practically anreadable) up to 1(X) (whiph is easy reading for any literate person); w/ = the variable 
word length in syllables per 1CK) wortlers (with its regre^ion weight of .85); 5/ ^ the variable of a\«r3ge sentence 
length in words (with its regression weight of 1.015); 20G.f34 - a constant which adjusts most readability scores to 
thr scale from 0 to 1(W. 

Actujily, the constant doesn't manage to put i?//jwnting on the 0 - 100 'icale. The Ftnwrai R&gisWr passage I 
showed you earlier yields h score of -7K which you c^n see is way off the scale. It has values of 233 syllables f^r 
100 words, and 154 words per scfitencc. That's pretty junusual, but I could actually have done even better. I spared 
you a sentence of 495 words lonq that \ found in a study of correspondence materials I completed rec^rtly, and 
that I will be r eferrtnn to again, I thmk you can see, at any rate, why such material qoes somewhat beyond the 
interpretation table's highest descriptors: 
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a table beneath it 



J, "vc*iv difficult" in !>tyJe, 

b. "si!tentific" in niagdzine level, tiruJ 

c. "aiHtjge" in reading level 

SHi<:t? icaddbility formuluv pruvHle <x)nve«^€nt indices of redding difficnlty, they have been very widely uiied 
in (III cirejs of eiiuc^Hon, mass communicatiwi, publishing, industry, and the military services. A numtjer of validity 
studies have also been dof>e in order to relate readability scores to reader behavior, paniculorty comprehensioi of 
written m.«terial. These studies are of two kinds. 

1 , Correlational -where readi^iility scores have been related to comprehension scores on large numbers of 
passages of diverse content. An exanif^e would be passages from a standa^di^ed reading test. 

2. li'perimental-where a particular i^ssa^ (or set of passages) tias oeen rewritten at one or several new 
It V fis of readability, to ^ what affect this h^ upon comprehension. 

Rather si rorisingly, the relaticNiship t^tween readability scores and comprehension has been far from perfect. 
Mdf'y re^isons for this are possible, of course: inadequate m^sure of ccwiiprehension, poor testing conditions, etc. 
But m an on going review of the literature, I have founo that two other factors are fret^uently responsible for the 
lack of relationship, or a low relationship. 

The first of these is failures to consider level of reader competence. By this 1 mean sucn things as reading 
skills, verbal ability, specific b^rkgruuiKJ for the material, etc. To give you a quick illustration, let me provide two 
passages. 

Adjustment of each of these transformers requres loosening the knurled nuts on the 
diametrically opposite screws which travel in helical slots in the cylindrical housing, and moving 
them along the slots with the fingers. If a tight fit makes this O|:^ralion difficult, the desired 
result IS facilitated by oscillatory rotation to the limits f^rmitted by the oscillatory rotation 
to the limits permitted by the width of the slots, accompanied by axial pressure in the direction 
of the required axial movement. 

The relative contribution of motivational and cognitive factors is difficult to ascertain 
since there is always a covariation of these variables in the humor stimulus. It should, however, 
be possible to arrange a lar^ set of humor stimuli so that cognitive and motivational aspects of 
these sti null are orthogonally represented (if such independence is not possible to create, that 
would be mte esting ton) 

The first passage, with a score of 23 on Flesch's Scale, is intended *or technicians ir# the Bell System. Most 
of the technicians aw high schoo* graduates; few have any more educatic n than that. The second passage, with a 
score of 1 1 , is intended for fellow psychologists by a psychologist wnirg about humor 

^h»s second passa^ may not, as one reader said, be funny. But it .s probably easier for its intended audi#»nce 
of psychologists than the first passage is for its audience of technicians. It seems safe to say, furthermore, that 
technicians would have more trouble reading the humor pas«;age than psychologists would have reading the trans- 
former f>assage. My point is simply this: when a passage is said to be readable, the statement should also say for 
whom It IS readable. 

The importance of this factor, reader competence, is easy enough to see, and the relationship is a logical on^, 
;it least within broad limits. The less competent the ^eader, the greater will be the effect of improve<i readability 
ufxin redder comprehension and reader ^>ersistence. Fhis is tiie case for reading skill, background, etc. 
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There is another factor, however, that is fiot so easily spotted in vaUdity studies. Let mc explain, using as an 
example j study I made recently of the reacWiihty fevels of corresfK>ndenc» materials. 



The United States Armed Forces institute, or US/^ FI, as you may know, is the c*iief '"corri^fWidence scfK>ol" 
for the Armetl Fortes. It provides curresfM>ndencfts n^tenals in academic subjects at primary, secorniary, and colle<^ 
levels, plus vocational materials at roi^ly hi^ school level. USAFI ask«i me to analyze tNj readability level of 30 
sets of their high school and tieginning collei^ materials (a set consists of one or more books fMus a study ^jide). 
I analyzed a rather extensive sample of materials-over 150,(XK) words -using a computer version of the Fle«:h 
Reaiiing Ease formula developed by several colleaj^s, cme of whom was Ijirry Stolurow, and I. 

USAFI research pm««inel Clay Brittain and Kim Smart then related the readability scores to the probability 
that students would Sf?nd in all of ^eir tesscms, whitdi is usiMiHy not the case. Suff iciwt course comfdetion data 
were available on 1 7 courses for such an analysts (there were insufficient data on 13 new torses. A rank orfjer 
correlation of .87 (p. < was found, hokJing length of counse (a criti«l variable) constant. The prodOct moment 
correlation coefficient an these same data yielded a value of .75 <P' -01). 



Here we can see that level of readability is so clearly related to the reader behavior of persistence. In a sense, 
of course. th<?r« nujst be a relationship to reader comjw^ension s^o, since the amount (XMnprehended v^uld go up 
if only because more material is read when readability is improve. The next step, ideally, is to be sure that ^e 
results of this correlational study can be followed up successfully with an experimental study, I have fwovided 
USAFI with versions of standard lesson materials in which the r^abtlity has been increased (i,e., improved), but 
with the content held constant. This experimental study of the relationship of comprehension ^ {^rsistence to 
reacfabihty has not been can led out as yet. 8ut even at this stage, the USAFt study shows that urxler the typical 
learning conditions in correspondence instruction, readability can play a significant role in reaBpr persistence. 

Now let me generalize briefly from these results. It seems probable that the chief underlyMg factor here is 
reader motivation. That is, the less highly motivated the reader is, within certain ftmitations or extremes, the more 
important readability is. I believe, therefore, that reader motivation as well as raider competence must be con- 
sidtfred when tivaluatimj the relationship of readability to reader tehavior. Here is a picture of the proposed 
relationship. 

A Mcvlel for tt>e Relationship of Read^ifilcy 
to ttie Behavior of Refers 



THC READER'S LEVEL 
OF PERFORMANCE 



IS A 



THE READER'S LEVEL 
OF COMPETENCE 



INTERACTING 
WITH 



(e.g., his comprehension, 
speed of reading, depth of 
reading, etc.) 



(e.g., his reading skills, 
verbal ability, specific 
background, etc.) 



THE READER'S LEVEL 
OF MOTIVATION 



INTERACTING 
WITH 



THE READABILITY 
LEVEL OF MATERIAL 



(e.g., his reading purpose- 
from "readinwi to learn" 
to "reading to forcjet," set, 
"stakes" etc,) 



(e.g., the reading grade 
level, word difficulty, S43n- 
tence difficulty, etc.) 



Why hasn't the effect of a factor as obvious as reader motivation been accountwi for in validity studies? I 

think I know one answer, lej^t Subjects arv m oxptTiments, cs|>ccjany if there is rirsirable rewards, tend to be 
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rath€f highly mot«v«i!e<f. Or, if they aren't already, most experimtmters try to raise the level of subjects' motivation. 
After all, an old dictum states thai sul)jects should be hi^iy motivated during experimental testing in order to get 
stables scoTt^s. 

Yet. tookal at »n the way I .im suiji|*fstHH|, <t appears that ex|»erimenters might actually tie mliicir^ the like- 
Uhooii of a significant relationship iietween read«ibiffty and comprehension. My on-yiing review of validity studies 
suggests exactly this. Most of my work involves the reading of microfilmed dissertations. Thoi^h they are bard on 
the eyes, they are a lot nwrc illuminating than journal articles, whidi sakiom jK^ovide critical details because of space 
restrictions. 

What I have noticed is that a combination of two sets of conditions commonly found in experimentation 
usually seem to reduce the likelihood of a significant relationship between readability and comprehenstw. CMe set 
of variables consists of motivation-raising coi>ditions« such as: 

a. having as the experimenter the subjects' regular instri»:tor; and/or 

b. carrying out the experiment durir^ a regular class period; arK)/or 

c. stating, or at least implying, that the study is of considerable im^rtance (i.e., will contribute to course 
grade, or help to determine the need for remedial work. etc.). 

The second set of variables consists of provision for increased motivation to have an effect upon performance, 
such as: 

a. providing generous reading time; an61oT 

b. allowing more than one reading of exptJiunental passa^; and/or 

c. permitting referral to the experimental f^sa^s during the testing period. 

Given, then, a combination of conditions which increase motivation, plus the opportunity for increased nrotiva- 
tion to affec* reading behavior, the likelihood of a significant relatic^iship betwt^nn readability and comprehension 
level appeared to decrease. 

We now have a dissertation being completed at Ohio University by C^rl Oenbow that apf»ars to demonstrate 
the motivation factor in another way. Two passa^ were rewritten so that for each there were t.vo levels of read- 
ability: easy (approximately yade 5-6) and hard (approximately 13-15 grade level, or college). Subjects read a 
version of a passa^, rated the content for interest to them, then took a ''dose" (deletion) comprehension test on 
the passage. One pasca^, which was on gun control, was rated significantly more interesting than the other, which 
was on wheat prices. The overall effect of readability was significant, but read^ility also bad a si^ificantly greater 
effect upon the comprehension of the non-preferred than the preferred passage. 

What are some implications of my little model of readability and reader performance for instructional 
materials? I can think of several. 

r The readability level of materials is especially important to the persistence and comprehension of 

readers of low competence or low motivation or both. Put conversely, readability is of relatively much 
less imfK)rtance tu readers of high competence and motivation. In fact, readability is not even very 
Important to a reader of moderate competence if his level of motivation is hi^ enough, as many 
tivera^ citizens demonstrate every April 15 or thereabouts. 

2. Tests of validity should be niade under typical learning contlitions, particulany as far as motivation is 
concerned. This may frequently call for a field type of test, when a laboratory type of test is instecd 
oftefi r^rruni out. It is entirely possible that the potential value of readability in correspondence 
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instruction mi^t not have becotfie apparent in a f.ibc^atory study » it did in the course comptetion 
study. It is similarly |K)ssiUe that oi^irar potentially valuable instructional variables may be falsely 
abandoned ^ non-si^if icant becau^ the conditions under which testing occurs do not represent 
weli the conditiof^ under ^idh learning typically ocairs. 

3, Finally (and less dearly), it is possible that certain variables in written instruction n^y be more effec- 
tive under conditions of low motivaticn. What tfie variables most effective under the condition of 
relatively high motivation mi^t be is not clear to me. (^haps organizing or thematic factors or test- 
like events (questions) migjit be candidates. But it is at least tecoming dearer that readability is most 
effective under the condition of relatively low motivation. 

Ai>d that condition, as in our correspondence study, is far more often typical during learning than we would 
like to think it is. 

i 

! 
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SUMMARS' AND IMPLiCATiONS FOR DAY THREE 

PeisonaliziKl md Open teaming Envirmm^ts 

Lawrence Stoiurow 
Profe^or of Education 
S.U.N-Y. at Stony Brook 



i will try to come to grip^ with important issues arising, in my opinion, from today's fxasentations* We 
have heard a nuoAier of excellent papers and f^ ofessor Yelon has already rented to them In a very interesting 
and stimulating way. Consequently, I will try to develop some of the ideas whidi were si^i^ed to me by the 
papers and discussions rather than retrace sfei^. Later we wilt p^epare a mare throuc^ and careful treatment 
of the finding, (Kobtems and their implications. R-ofe^or Harriswi and i plan to sumnwrize by identifying 
poin^ raised by symf^sium papers and their implications to sugc^t some possible directions which NtE might 
program in its development of educational tectmolc^y. 

It is dear to all of you, as it is to me, that many useful contributions were made here today and that 
there is a great deal of food for thou^t in the pai^s presented. Today's presentatiorts considered in the back- 
ground of the previous two days make it clear that the tedinoli^ used in hi^er ediK^tic^ has worked, is 

working and can be made to work even more efficiently, extensively and acceptably to both students and faculty. 

« 

it also is clear that it not appropriate to talk about a technology any ledger. We have heard about 
several viable educational technologies each of which, while developed as a separate effort, ap^^rs to be capa- 
ble of coexisting, with advantage, in a single institution along with the others. The Integrated use of the 
several technologies would represent the next logical step in the evolutionary process traced by the three days. 
It wcHild permit students to select from a wider range of possible pedagogies according to tfieir preference 
and demonstrated value along the general lines Si^i^sted by Professor Lumsden's ''free ped^ogk:al economy/' 

It also is apparent that each of the technologies is deaiirg effectively with a set of prc^ems and students. 
What IS not always apparent is that the most difficult fxoblems with which they deal are those that have 
plagued education throughout its history; these are the problems of teaching in efficient ways so as to produce 
learning and the desire to learn. An imfK}rtant historical difference between old and new approaches and one 
that has given impetus to the search for new and more efficient ways of teaching is the growth in the numbers 
of students to be accommodated by an ^uuilional system. In the more recent past, increases in numbers meant 
a search for ways in which the system could deal with larger and larc^r groups. Some of the technologies are 
designed to do this in relatively simple ways such as voice amplification to permit a teacher to reach a larger 
number of students. Other technologies, however, for example computer-aided instruction, are designed to 
increase the numbers of student related to while also personalizing their experience in the learning environment. 

One thing that the new technologies have done that was not done earlier is to make the basic problems of 
education more apparent to us. While this is a very useful contribution, it does not come without cost. The cost 
IS that the problems are thought to be those of the technology, produced by the technology and peculiar to it. 
The fact is that they are problems which are indigenous to education and basic to the process of education it- 
self whether, or not, technology -is used. 
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One reason vvJiy trikjcxitiofu! tt^nufogv has jfJumtnated the b^fc urobfems of teaching and learning is that 
it mmimi/es the humjn fdctor m the itelivery prucess. In order to mtnimiate or eliminate the human compcment in 
ikMfvery tt ii necessjiry to mjke very explicit just «4iat is necessary and suf f tcient to achiewf a set of instructionat 
objectives. Everything hds to be careftifly planneii in detail prior to the delivery of the instruction to the student. 
Thinqs can nor Iw h*H to th*j fe»ich*'t^ <fnj4ination amt imfHOvtsatinn m the spot tjecause there is no one present 
dt the tinw? to iniaqim? and Hnpfuvist. In t^lucationjl technofo<jv what yuu si'h m development h what you get m 
dt»lfvefV. 

When It IS pussibit? to teach without a teacher the failures of the pi ocess to produce learning or the resulting 
tnefffCMrncies *fte clearly those of the proaess of de^wei y that is used. On the other hand, when a teacfier is the 
source of delivery, it is not clear where the pt obiems are. The method, or strategy, t<sfHf in the teaching program 
may be good, bt*t n may not have been prc^ly employed and other thin^ may have t^n added fnadvertently 
by the teacher that diter or retluce the effect it has. Oie major area of interest m this re^rd which is, as yet 
unresolved, is the contribution of the te«:tier's personality to the effectiveness of the instructiwal experience. 
With the capacity to ctoliver instruction without the pre^jnce of the tearher, research on the relative effectiveness 
of mstiuctional methods and strategies becomes possitrte in a way that allows tor tfte presence and absence of a 
teacher ami of teachers with diffensnl personality characteristics. The technology permits the introduction of a 
set of comparative conditions not previously available to the educational researdier, 

Ttte mtertJependence of the tlevelopmental process by means of which instructional materials and delivery 
plans art- designi-d and develof>ed differ ^rnonq human and tiKihndogical systems. In human systems the inter- 
ttependence is so great that thuy can not be separated without great effort and this limits the ctegree of scientific 
ifutiiff y that IS possible. V^f:n^ is no way one can determine the relative contribution of the desic^ from the 
<teliverv of instruction unles> the o^iacity to tteliver one desiffi in a variety of ways and by a variety of nw^ns 
tl>at can be sfwcified and replicated exists. While it Wi» always possible tor a human teacher to separately plan an 
instructional interaction in detail sufficient to f^rmit someone else to carry it out mlb comparable conditions 
prevailing and critical conditions maintained, this has not t^n setiously considered for a number of important 
reasons. As a result the developmental technology for instruction has not emeryetJ prior to the development of 
viable delivery systems. With traditional approaches, reliance is, and has bt^n, f^aced upon knowledge of subject 
matter and the intuition of the teacher. Educational technolt^ in all of its forms of delivery, as revealed in this 
symposium, h^ formuli^ted and used a develc^mental technology of sonte kind. Geneially this has meant that 
technical specialist are involved so that the develofKnental process is more involved with respect to f^rsonnel 
as well as techniques than it is when accomplished by conventional means. 

There is no doubt that delivery technology has captured the imaginaiion and interest of a lar^r group than 
has developmental technology. Therp ^tv a nufnt>er of re^ohs for this. One is that the delivery tecfmology for 
instruction is sliaicd by the entertainment, business and scientific communities. This sharing has had its spod and 
bad sides for all of the educational technolgies. The tedmology for ctevdcn^rr^t, on the other hand, is umqae to 
the instructional process althoui^i that has not been as clearly recognised as it might be, A strong »id self validating 
technology is requireti for the ttevefopment of mstructional systems and materials. Brian Lewis has provided us 
with some inti resting and liseful guidance here 

The tL'ch^ologv ot dt}velopmffnt was probably most developed in tho presentation l^y Bnan Lewis and it was 
elabordtetl upon .v, Hrotessor Yelon. These two excellent presentations indicate that this technology is the applica- 
tion of the lM?havioraI and social science theory and findings to the dirsiqn of learning environments, and not an 
isolated development, Whrn properly practiced it is a carefully planned activity requirinr^ a variety of specialists and 
Its own technologii:al capahilifies ail of which are different from thos^ required for delivery. Professor Klare's presen- 
tation also brou<iht out, dlthoiigh it did not delve deeply into the point, thdt there is a need and an emerging 
response to that ne^d. It is th^' need for new tools and methods for materials development. A fairly technical area is 
emeti^nq to meet this need. It is designed to provide authors and instructional-materials dL'Signers with computer 
capabilities that can make the process of ortgmal materials development dnd their re^lnsign more cost effstctive, 
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rehabf<!, ami v»iluJ. Hit? cvMiHHJtur i!> Immq ift;velop«<1 as an infofmation proces^r, not as a number crunchef . This 
area uf icchnotoqv coulti descritjwi as appiftHi tnlucationat psycHoiirif^^tstics. ft coukl liecome a substantial area 
of !H)eciali/alion $n th« n*»ar future if iKoperJy gutifetf and suff icientiy fuiuhsd. 

Vw ottSi of d|>{*i<efl (MJtiriifif?Tial psycholimiiiistic? wiH r^qutit? a wtiuli; anay of computer software capabiti 
tii^ 111 an ifiteyiatvd enviroiuntfiit tuiiuitHl lor tile author, instructfonal ma teriais-^and systems desi^ers, and the 
tniucatumai researcher Si.i«N:(aii/tiYg m imtructionat pragmatic and m tht? thec^y of instruction. In addition to the 
many i*Hi&tfny editorial and text processing capabilities and the ability of computers to set ty^w, a variety of 
aoalyticai software is txjiiiy developed^ and it will be expamfed upon as authors and instructional ctesigners learn 
their traile and define new requirements. Readability analysers, synonym banks, parsers wiHi targeted areas of 
duplication, key word m context analyzers, disan^biguation systems and indexing software alt represent hits and 
pieces of a com^iuter-tiased system needed to serve the educational industry. Furthermore, it could t» develi^ied 
at this time to serve with advantage the needs of instructional systems designers and authors for all of the technoic^es 

Professor Hanson referred to computer-managed instructicm (CMIK This conception of a management system 
that pioviili'i d user with tij^id and convenient access td bits and pieces. of informatiim or prace?;sing capabilities so 
thiit they interrelate with <^e another is an important concept with respect to the use of computers in f»rsonalized 
and oper^ learning environments, T»»is conception was not developed for either the delivery or the deveiopment 
processes fdCfr<l by the institutions such as the Chicagor TV College, the Open University and the others reported in 
to<fciy's session. It, if am^licd to ttiese ^vironments, could tje an enriching and facilitating addition to what tftey 
have and doing so would represent the next stage of their l;volution artd alluded to earlier. 

Relateii to The CM I conception is its counterpart for the developmental technology of instructional materials 
and systems. I refer to this counter part notion as computer-managed authoring, or OWA, CMA could turn out to 
bf! one of the most significant developments of the seventies, and if it is not then the other technologies will not 
grow at the rate and to the level of efficiency that is poi^itite and essential lor the rapid spread of ^Jucational tech- 
nology, and the reali/atiod of its economic ijenefits. A well designed authoring system with a^ipropriate capabilities 
woidd ifc able to assist an author or instructional materials or systems dess^er in the ctevelopirwnt of a wide variety 
of different forms of pedagogy and delivery. It would assist in the reforming of informatiortal data teases to identify 
" objectives and to relate the statements of objectives to forms of assessment, It v^uid assist in putting t6^ther a 
seen jf 10 for TV, a script for an audio tape or film. It would g^^-wate ^aphics ai>d permit their rotation and perturba- 
turn for selecti<Ki of displays in support of information and concepts in instructional dialo^ies, simulations or in the 
fMi?paration of slides for ifse offline as vwH as online. Not only would the CMA software supfK>rt the manipulation 
ami gpnifrating capabilities by aiding authors in the production of materials for instr^icticMi, it also would provide a 
capability for storing units., taggjng them and later retrieving them so they could be recombined into a variety of 
relationships with other materials to form whole xhjw sets of instructicnial materials to meet unanticipated needs. 

We {lid not hear as much about the devf?lopment and use of computer-based system today as we mi^t have 
thouglit. The personalization of instruction m an unbounded learning environment is an ideal set of conditions for 
the realization of the potential of CAl, Inoadly conceived and developed. Furthernrore, with CAI the level of 
detail required in planning and the ranr^ of variation possible in instriKrtional interactions is greater than it is for 
any of the other forms of instructional technology, Howe^r, I feel that many are not ready, nor do they feel the 
time <s npe, to mdke the leap. They may In? right, but someone has to be working within the existing centers in 
which technology is being successfully employed to develop the computer capabilities that will move the entire 
enterprise to its next level, 

CAI IS an mterestinq technology to ^ufsm* at this time since it t^ars a very supportive relationship to all the 
other (fell very technologies, and to CM I and CMA. It might \ye useful To look at some aspects of the relationships, 
fn the area of development of materials CAI is probably the most demanding on the author or materials develop- 
rnt?nt team because of the level of instructional detail it requires, f^rticulariy if the full rmge of delivery mocies is 
used as appropriate for the accomplishment of a variety of Sf>ecific objectives over the long haul. To a greater 
extent TV, films and even autin tntofial Technology allow nreater freedom to the developer of materials than does 
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CAL This tiuih its guoif and ti^ii %ule%. Oii the good side it is tmiHiftcint to point out that the ri^r and detail 
required for C Al materUhj developnie»t could make a substantial contribution to the other technotogies by defin- 
ing and specdymg varwblefi amd conda«)m thai shoafd be attended to under other conditions of delivery as ^mll. 
The requirement for objectivity and enplicitness m planning an instrufcttonal exi:»riem^ or a learning enirirortment 
Ixoduct'S ducumentatiun that can Mjr vt* as a niudt*! vvhtch is eitlier ^ihareil or used as a point of. departure in the 
deveici^ment of materials for otlier technologies. Thus both the diffusion of infomuition and the vatidaiion of con- 
cepts and techniques for both design and itehvery become better and more eff lively realized. While not 
apprccfa ted widely, the cost of developing an hour of CAI is less than it is for ^ hour of most of the other tech- 
nolo^tes and significantly lower than that tor TV or filrn^ Not only that, but also the cost curves differ. The 
<kvelo,miental cost for CAI has been coming liovwn vs^Jle that for the other technofogies has i^en rising over the 
Slime period of time. On the t^ad st<te, CAi delivery costs are greater than they are for the other tedinofogies, 
expeciafiy f ihn and TV. We need to consider the total system when wre deaf with the question of prciductivity. 

Since CAI material! require greater detail and more complete analysis than is the ca^ for the other technolo- 
gies and Since this couid l»? turned to an advantage for the educational incfustry by providir^ guH^nc^ to the design 
and delivery forms useil by ail the instructional technolc^es, it v^uM seem that CAI has a special place in the %et of 
technologies we have been tfe-^ling with. This point is supported further by the fact thai CAI systems generate more 
specific effectiveness data than do the other tecftnofogies. CAI is an interactive technology that records interactions 
in detail^ and in a form that is retreivaWe and processabrfe by computers fm:ihtating the formative and summative 
evaluation processes, it is important, for these reascms alone, to consider CAi in terms of a long, as welt as a ^ort 
term investment. As an investment within the set of technologies it has many more fiK:ets of potential significance 
and value to the improvement of instruction and learning environments in general, than the others. It not only 
produces instruction in the short term and with a degree of effectiveness comparable to the others, if not superior 
under some conditions, it also can be used in Wfays that are supportive of the otfier technologies arnf their growth, 
tt has the further advantage of costing less to produce. When it exists at least two ottier impcn^tant things also 
become rm>re feasible. One is the develcH^ment of information atxiut devefopmental technt^os^ for instructional 
materials and systems in ^nerat. Another is researdi findinc^ which increases cmr understanding of the process of 
f ^struction, the ck^si^ of learning environments and the research methoddogy required to furttier these ends. 

Many feel that the Jevet of detail required to itevetop CAI materials and systems is not warranted by the level 
of existing knowledi^. This seetrft to be circular argument tiased upon a shortlisted view of the state of instruc- 
tional research and the scientific process. Mew tools for scientific inquiry have always opened up new areas of 
discovery and <leepened our understanding of the r^turai phenomena stud^. instructional systeoB are now avaiia^ 
ble that permit detailed control of ledrnnig environments and allow for the manipulation of fwsonalized conditions 
for the purpose of studying interesting arul useful statistical interacticm effects with methods and strategies of 
instruction. The fact that previous research has identified little that vi^ously supports a theoretical position like 
that proposed in my idiographic model, for example, does not dissuatte rne from the petition that research on 
statistical inter«:tion effects mvolvir^ personal diaracteristics of the l^mer and the Sf>ecific nature of pro-ams of 
instruction are a useful and appropriate area of research which justify the development and use of CAI systems. 
Without the computer to manage the conditions and the independent variables it is difficult to do research in this 
imjKJrtant area so pssic to the personalization of learning environments. Thus the fact that there is so little existing 
data does not seem surirrising nor should it be unexpected. 

The basic problems tacirig educational technology are the very problems that face education generally. These 
are the problems of learning and instruction. Personalizattc^i of instruction if it is to be realized and cultivated 
requires research and the level and scale of the research has to lie different from that with which we are familiar. 
Also the capabilities of CAI systems are unique so one needs a system to study their implicdtions. Consequently, 
the ifwestment into educational research has to t)e reassessed. Many feel that the educational industry seems to 
havff experienced too little too late in the sixties. I am not convifKed of that at all, and feel that neither should 
you since the things we have heard at>out today and on the prwious days are the fruits of those investments. In 
this light the problf^m with difftision and fH?rceptfon seem to me to lie more with the expectations than with the 
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results. The fttiitratHm v^ith Idck of proyrew ami discoura^ment in the tack of Support which many claim to feel, 
seems to be the result mofe of what was expected thaii the leafittes of what was delivered, »f we consider what 
ct uld have been ddi ver ed with the limituti Knowledge base, the tiewness of every one of t. teChudogies and the 
scarcity of top quaHty personnel the result has faecii very good Mtdeed. In my opinion, the reports we have heard 
Htc most ehcouraqimi and Sfioak vfiy vvi-U <«f tin* ti nits of a tiuMift-t investment relative to the si?e of the problem 
anti th<! short time available to i.*ulnvji«' filncational technology when it is just coming of age. 

The decades of the fifties iinti the sixties have f»odiice<l educational technologies that not only deliver at 
compeittive levels with the traditional moiles in many aicas of hi^er education, and especially in professicmal 
education, they also provide larger nunilms, if not new ciroups, with qualitv instruction. New se^nts of the 
adult {population are Ueing reactiwj iiy t«k!*isioit in this country and abroad These students are presentir^ new and 
eiiually interesting protilems to institutions. The estaWished institutions are being c»»llenged fay this student grcwp 
much as the returning veterans challen^d the colleges and universities in the fifties. In retrospect many of tlw 
academics who comi ilalned at the time that the veterans were in collets are now i(K)king with nostalgia at tl» 
lime when the student veteran was camiius as an ea^r and dedicated stuitent. The reports by {wfe^r^ Zis^ll 
and I ewis have sug^ted that this may happen again with another population of stucfents. From the resear^ 
point of view the returning adult, who after several years itecides to get an education or to complete a degree 
program represents a segment of the student |x>pulation about whrch we have very Uttle data. We therefore ha^ ^ 
to proceed cautiously in extending the knowledge we now have which is so heavily based upon the typical college 
freshman or sophomore. This new group represents an im»Mrtant research area for an NIE prof^am. 

The basic problems of teaching and leaming have to be considered whenever an ed(K;dtMmdl initttiition 
refales to students. The nature of the student population can be an important factor in cfetermming the results 
obtained from partrcuJar approadies and conditions of teaming. The different technologies provide educational 
researcher with different cai^biMties in dealing with basic issues about learning and instruction not only do ttey 
contribute in a scientific nranner to this end, imt also they p^mH research m w¥€k Scientificdity they fmnnde a 
way of stabilizing the conditions of learnmg to make them more replicable, and they pormit tlw manipulation of 
variables. Furthermore, they aid in the collection of data #n ''r^l time" - while the student is learning rather 
than restricting research to the collection of data after the fact. 

Unfortunately we do m)t have much of the data need^ to make valid d^isions abmit the nature of the 
<5ptimijm conditions fur a learning environment to be used with diff^ent students. We do have many of the 
mifthods .md tools for identifying critical student characteristics to study a number of research questior^ relating 
to pf?rsonali/at«on. We know much less than we would like abtwjt how to ^ different kinds of stucter>ts to learn 
efficiently af id how to get them to like learning. To a great extent the lack of avaitabte tools is at ^ t>ottom of 
thesi' pr ofjlems. With conventional approacfifs to itistruction, the collection of data represents a sut^ant^l 
additional undertaking with its cost add-ons. While it is still a sis^iificant undo'taking whmi instructional teclwolo- 
qv is available, it is nevertheless true that with CAI at least tti» additional cost of prcKfucing res^rch data is not 
proportionally as great nor the task as difficult. The reason is that the basic investment in the dei^k^frmit of 
materials and systems automatically produces the data needled as a byproduct if the CAI system is fwoperly fHit 
together in the first place. The additional input that is most required no matter what c^mext is involved is in the 
research i^siqn and the implications it has for the numbers of students required. Compute technology does not 
help here. This discussion is intended to underscore the point previously made that the educatioral technologies 
are not just instruments for delivery and develofwnent, they also are tools useful in the production of nm knowl- 
edge ^hout leaching dnd learning. They are in research tools. 

An imfwtant question, not addressed directly by the papers j^esented today, but one to whidi we all 
should be sensitive is how make each technolo^ more of a hand maiden, a tetter tool and eventually a well 
honed instrument for research on the instructional prc^ess. In dealir^ with this set of problems we are not just 
dealing with formative evaluation which v^«; so nicely developed by Professor Yelon but also with summative 
evaluation and the approaches to it provide by quasi-expa-imental desicKis as well as more rigorous experimental 
(fesigns. 
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Ideally the technology when used tor research as w-ett as f of dclivei y of iintf uction should not intrudti mi 
the process of instruction itself, but rather serve as an aid in the solution of insf ruct^ooal prohtems. An important 
diffemnce m our present viewiHjint from that which i<ve held in the past when teachers we^e the only avatlabte 
source ol inslroction is that thi^ system of delivery should km a catalyst^ fi^Uitatt^^S the leamiTig process without 
entering into the process itself. In this sytnf>t)S'um wc havit not taikofi alniut obtru-^ive and ungbtrus»ve lechnology. 
i think It is correct to siy, however, that all of us are interesiecS in seeiivq that the new iiiSifui;tioiia{ technorogy 
as unobtrusive as possible. One way to do this is by making the content and the prtKess of inforntatibn and per- 
ceptual experience more apparent than the delivery system itself. The delivery 5ysti>;it techrtotogy. whether it «s 
TV^ AV, At or cctrnputer mediated interactions, should blend into the backgroisfd of the learning environment. 
J should not compete for the student's attention with what it is that the stuikmt should be learning. Neither 
should its denrwntf for •earning m w skills be eKcei^iw^. 

There are many different ways to look at the percwttons of educational technology as It relates to dn 
educational institution. The purpose in doing this is to dcjvelt^ a persprctive that permits us to make a dfcision^ 
about tfHJ altimiatives hi a consistent frartte of refo^enc^. The pai^s presented today indiate that there is a con- 
siderable rar^ of variatioft in the capital investment retiusred, the typ^ of resources needed and in the ext^rcise 
necessary. Video tape technotcKiy and the mailing of cassetts reptesepts a lower level of investment in facilities 
than does the use pf broadcast television with electronic delivery. Pr ufessor Baldwin has kindly ^ared with us some 
cost data of a video cassetts and 8r?an Lewis, l*rofe&sor Zigarell information about television, in spite of significant 
differences, it was quite cteir that each technoti^ can become an effective apd impressive approach either within 
an existing institution, or in a separately developed institution. The costs ^nvoitved differ not only with the mode 
of delivery and the capiiai investment but also with the kinds of per^nel requirt-d 

While each techncrfogv can f>ecome standard practice in an institution, the question of effective transfer to 
new institutions remains an unsolved problem. Not dealt with here is the question of how an institution that 
wants to emulate an established pattern actually ^ts started. The documwtation provided could be helpful in the 
early stages. The successful institutional efforts about which we heard are not themselves equipped to assist others 
to geit started in any substantial way if they want to establish a comjja'able capability- This probltm should be 
faced by fuiidinii agencies. 

The widespread use of educational technologies in other institutions raises sevef at qu«5stions which in the 
context of this symposium on productivity in higher < ^location are sii^nificant. First, it is im yjrtant to point out 
that the cost ol each of these examples does not necessarily have to justifieci on the basis of transfer to other 
institutions. The cost of developing each of these centers probably can justified on the basis of its use in the 
institution in which it has grown and developed. It is interesting, as f^ofessor Yelton has pointed out, that the 
number of students related to by telrvision technology represents a growth curve that can not be matched by a 
university with walls. 

This point raises another Interesting question regarding thts cost analysis that can be made of educaticnat 
systems. It is the question of the cost of adding additional students. Audio tapes, television and CAI alt can add 
lar^ numbers of additiona! students without incurring a major step-function in their cost curves, whereas a 
university with walls does incur a step functton with relatively amall amounts of growth. While the average cost per 
student may not differ when alternative apfxoadies are compared over a particular range in the ^\ze of the student 
population they sen^e, it is still fx>s3ible that the alternatives could difff?f significantly in the periodicity and 
magnitude of the additional capital investment to accommodate yowth in student population. 

Another aspect of the cost problem is the inequity in the burden of proof. The new approach is in a defen- 
sive jKisture. Each of the presentations we heard totlay has had to be mwe co^i conscious and more defensive 
about its costs than tfie convwtional institutional arrangement which it replaces or happens to be compared to. 
It frequency, thoi^h inappropriately, is assumed that what is now l>etng dow is cost effective and thi? btfst that 
can be done with existing resources. Any discussion of alternatives to the establish^ practice often ehcds a defen 
sive posture rathrr than a willmqnrss to study ami oxplorf^ the atternal»ves in their terms and in relation to 
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qufsstion^ uf i^rociyctivity or rt^tetive benefits. This detersive fxjsiure often is manifest i^s an off^stve posture amJ 
the attack is f r equ^^ntly focused on the very auctions whteh are hard^ to answer by tha ctm^nttonai system 
Itself, namely, how effective is it. Often this argument is made on the basis of inJcKmation relating to the input and 
not on the basis of the cnitput {effects m% studc^tsK (^tainty the input analysis as described by &ian Lewis leaves 
httie lioiibt about the care and detail of the tnfor maiion analysis engaged in when n^terials are developed for use 
in the British Open University. Fe^ conventional systems, if any. can provide the equivalent information and 
documentation regardti^ the input to itr courses. When we can not measure the output or wf^n we can oniy 
measure some smaM aspect of it, we rely <wi the care and the type of f^ocedure used to {^oifcice; iearnins-tfffecls. 
If a conventional system hires teachers wttft the rt#it credentials and uses the Hj^t texttooks it is assumed that {t 
is doing a ^>od job whether or not there is any objective data on the (^formance of its studmts. When diff eient 
iwpcedures are used and when video tapes or computers are employed to deltwr tfW instructitm, for examine, 
then output measures are required, but they stilt may be hefd suspect by the established institutions, ff some of the 
measures of student effects a«e new, then there is a tong process of credenttatif^ of the instruments and prorc* 
dures that ha^ to be en^^ in. Ttiey tiave to validated, *elr reliability has to be estaltrilshed wd the Madwiic 
peer gro has to be convinced that the instniments are, in f^t, producing u^t and meaningful information. 
There is an additional f^i^tem, hov^^v^. fl is tha* tf«J connparison with the conventionat system ts diffit^ilt, or 
impossible to make for any one, or mwe, of a lar^ number of reastKis, This puts things Into linrtio with respdct to 
adoption and use of the new ap|»ofiM:h and the staying powfr is on tte sicfe of the conventional system sine© it is 
already funded and has its sources of funds rattier well established and the mechanisms for obtaining ttwn well 
developed. t ' ^ 

Still another manifestation of the inequity in the burden of ixoof comes when th^ d^t^ are available frimrf a 
summattve evaluation in whfch two or more, instructional systems are compared. We experienced smne dii^sstons 
during the sym|x>sium regarding the stirfies f^roducing ''no statisttcally si^iif icant differents/' Many, if not most, 
studies comparing conventior^i <«iiU a new alternative have f^oduoed results indicating no sigrtificant ciiflemnce lr> 
the performance of the students. This finding should be viewed in perspc^ive, but it usually is not. Tliese studies 
compare a new technolc^ which is relatively young and not developed with an established alt^native that 
has reached its plateau of growrth. I^oper peispective can be achieved when we view the new technok}gy in rela- 
tion to its growth cycle to determine where it is at tf>e time the comparistHi was m^. Snc© tfm technological 
approach is typiiratly at an early point in its developmental cycle and the osnventional a(^>roach is typtcaHy at 
maturity, the co|nparison i^ like that of a child to an adult in getting a task done where maturity plays an 
important part- When viewed in this way the "no sig!!^fi<3nt difference** result is, in fact, encouraging, not a 
disappointment. 

Consider the two kinds of error we can make in rraking inferences based upon data relating to variables |»^- 
sunwlj^y affecting the instructional environment. One kirKf ^is to reject hypotheses that are useful; the other is to 
accept hypotheses that ^si « i*ot u^fui. The selection of the minimum confidence level and tf^ stat^k^i te^ used 
affect the relative likelihood of th«^ two kinds of error. It vrauid seem aprn^c^xiate to develop guictelirm for this 
research pr<*tem. At this point in time and given tte nature of the inforrruiticMi base and the status of instructional 
theory, it would seem that the i^etter strategy to use would to y^actices that may result in our acceptir^ 
more false hypotheses than we re^t useful ernes. Oie reastm is that when research is dcKie m Wvo the c<»t is 
not as great as it is when research is conducted separately. Therefore the rast of further t^ing wmild not be ^ 
great. Most of the cost of research on instruction conducted in vivo would be ap|[^opriatety assigrmi to the cost of 
instruction itself since stuctents would, in fact, be tw^t anyway. The ar^ment is bolstered further by the fact 
that in conventional instruction there is a wide range of variation from instnictor to instnjctor anyway ard ewn 
from time to tirne for a single instructor. Thw^efore the testing of hypothes^ that o$e weH cotu^i^ aiKl carefully 
introduced would be likely to generate no more variation in the conditions of edu<»tion than now occurs. Furth^- 
more, the hypotheses tested would t>e i^n^ ated to improve the level of effectiveness of the learning environment 
so the spirit with which the variations would be prtNJuced would have the same kinds of expectaticn? as any 
teacher who was trying to impH'ove a learning environment based upon his, or her, intuition. 
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The daw processing modtils u^ni in collecting and evaluating cfati* to make deciskms about alternatives In 
the ikdesign of teaming envir onmems can make an important difference in the growth anc) tleveiopment of a 
fittW Wust of the research cunductetl thus far h«is relied on ctescriptive and mferentJal stdtt$ticaf fiKxtefe tn whtcfi 
comiKiri^ns of central tenitencics or vdrtabk^ are ittvolveii. The problems requireing deccsion however involve 
tnofe infarmatiun* important are the c^ta on uKiilies and cost^. If would seem appffCHiriate/therefore, to develop 
a proiect that fostered the use of ctecision theoretjc models and the developmmt of method^ and instruments to 
fH^oduce data relating to utilities and costs. This should be a ht^ priority effort within NIE. it could benefit from 
the appln^tions of these moifeis in ottKjr s^ors of public interest such as the environmental area where subjective 
utilities are being used* Consisteni with this orientation is the use of newly develc^^ testing models to determine 
the student's confidence m his own information relative to a set of problems he is given to solve. This is sometimes 
called confidence testing. As a set of conditions thisafH^roach to evaluation ;tncrease$ the reliabitity of a test 
rather substantially, ft alio has the advantage of pruviding more informat tori. for tK)th formative evaluation and 
student evaluation. Professor tumsden's recommendation that education be restruaured so as to provide students 
with pedagogical options wiHitd if implemented produce data on student prefer'ences for optional rcHJtes to an 
educational goat This is consistent with what is being prc^K^sed here. I would go further and su^^t that the 
system be desi^ied to allow for the roltection of <^a on the f^eferences of others as well. We nml to knovi? what 
the tewhers prefer to vw>rk witft, what the parent and taxpayer (»^efers« ami what the adminislrat^Ks and mana^s 
prefer. It is conceivable that these mi^t differ. If so then there is a problem to be fa«d m attempting to optimize 
the situation for the ^t of individuals involved in making the system work. It would not be useful to set our 
si^ts on the maximization of the student's preferences if it resulted in a minimiaation of all the other preferences 
involved. 

The fact that there has not been a more extensive use of decision theoretic models is not an oversi^t or 
accident, in r^art it is historically determined since these are relatively recent developments^ but possibly more 
important is the fact that ^he instruments and procedures have not t^en devetoped to produce the kinds of data 
needed to use these models. A difficult problem is the desi^ of situations so that preferences can be measure. 
Anoth^ is the collection of subjective utilities; a third is ojst in f inartcial terms, titme of the mxounting syslerns has 
been developed to the level that permits these morieis to be used with m<iximM[m effectiven^^ Howev^, this 
should not be used as an excuse to ignore their potential contribution. On the contrary the models, if we are corv 
vinced of their ultimate value^. should be used to di^termir^ the nature and priority of a methottologioi} researcfi 
and development effort to provide the teachnic^ies arri tCK>ls. 

A further complication worth noting is the fact that costs and utilities vary over time. It is nece^ry there- 
fore to plan the devetopmeni of procedures that are cumulative and ftistorical. We ne^ to develop a>st curves and 
utility curves arui relate these to other cor^iitior^ and variatTles. Decisions rmde at any point in time take a while 
to become effective. We need to know what the conditions are likely to be '.vhcn they tecome an actuality. The 
actual costs and today's estimates of utility may not t% an accurate irKiex of what they will wf^n a r^w technolo- 
gy is intrcxiuced in the future. To use decision models with cost data it is important to develc^ the data b^ for 
use in forecasting. Comparisons of labor intensive apr^c^hes with technological alternatives requires attention to 
the curves. The cost of labor is going up and the cost of hardware is ^ing domi. But the costs are not just ttie direct 
costs or current expenditures. They also, in the ca^ of labor, ir^lude continuir^ a>sts asst^iated with ber^i^, 
especially pensions. The recently publicized budget of the city of New York, for example^ produced consideraNe 
comment relating to the item on retirement pay. It is quite clear that we need to consider the use of personnel 
resources in terms of the percenta;^ of the total costs and in relation to their need to achieve educational goals. We 
must define need in a differtjiii manner than in the past sinc^ there are obvious technical solutions to the delivery 
and development problems which did not exist before. We have to include in the definition of n^ the preferences 
and subjective utilities of the students and the otl^r parties in the educatior^l system. The siuuent population to 
t>e considered in planning and operating an educational system is becoming quite differmit ar^i the trends in the 
nature of the student population are already clearly different from the recent past when tfw student population in 
higher education came predominantly from the graduating high-schocrf student group. As technology of the kind we 
have heard about today frrcfeases m its development and availability the kinds of students and the numbers of 
•itiKlt iM*; to he r*rlatf?d to will chanqi* dramatically. This also means that either nrw institutions are to be formr>»< to 
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accommodate the different educational ne«nfe oi the present oties will twe to mwiify iheir programs and pw^oce- 
dures to dccommodate d wider range of students arnl student needs, Proiiabty both wtU hap{;N3n if we judge on the 
bdSfS of what we have imani hefe at the symposium* 

It is. paraifoKicat ttiat we swni to buf at a new frontier in education just at a time wtien tfie financtai resources 
jre pro^Kirtfonatty dryinci up. This means that we are faced with an economic condition that is going to force 
change Init that we will have to (proceed with care and a yeat deal of ptanning to a>nserve the scarce resources 
availabfe to try and to eKiriore alternative solutions. It would seem that the technology i^v lelate to this mix of 
condittons in a useful way. It can, for example, permit the mcKleling aiKl simulation of alternatives. As a method 
of planning this may provide the need^ guidance as welt as cost benefits. 
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EDUCATIONAL TECHNOLOGIES: RECmflMENDATIONS FOR 
RESEARCH AND DEVELOPMENT 

Lawrence M. Stolurow 
and 

Shetley A. Harrison 

The Symposjum Format 

This symposium was i^anncd to exptore« in some depth, the pntpnflal and tmptications of using and 
advanced communicaiions technoIogy4^$ed systeim for fmproving the ofganizationat effectiveness and opera* 
tionaJ efficiency of hi#ier ^ucatioiiai institutions. To acfXimpMsh this, s{K^ers were invi^ ami thdr papers 
commissioned. Eacti one was asked to pv^nt information aboMt the technoloipcat area with which they were 
most fdmiltar and in «^ich they were t^rrentiy active. 

' The seminar, consisting of an evariirtg session on Sunday and three fuU days and two evenings^ examined a 
ran^ of aUerrtativ^ for ina^sing the i^oductivity of higher educ^tionaf institutions with respect to instructional 
technofogy. It seems to the planners that the heart of the problem and tfie most critical fumriion of institutiof^ 
of hii^er education is instruction. For ^rs re^on that process was the primary focus of attention. 

The invited experts were free to ei^r (X^esent the»^ prepare papet or to defMfmi upon the grmip to have 
read the paper in whidt case they couid spend the scheduled time to elaborate or further develop and editori^ize. 
Discussants were identified to comment upon the pap^s and to provide a oritsc^ as weft as to devetop imprfica* 
trom tKid useful elaborations. In some cases ix^oblems w^ identified wid difficulties pointed out. In additi<m« 
the assembled group were given opportunities to discus and develop points relating to the presentations. 

in planning the symposium a matrix representation was developed and used. We re|ected rather eariy a 
set of rubrics that related to media and to tyf^s of systems. Our rationale was simply that the re^ problam were 
triose that could be referred to as the destpi and development of learning envirmiments for stixfents in higher 
education. Media and systems were to be considered as tools and not as the defining condition. The three-by-four 
n^trix gave structure, focus and operational guidam^ both to the symposium it^lf and to this report. We fouml 
that the participants when invito and introdt^ed to the conception r^pond^ positivdy. Thore wbs& no argu- 
ment or negative feeling expressed so we are led to brieve that wfiite untwivent'orwrt, thp structure did not 
inhibit or distort the ^alyses we sot^t and the oxistructive de^opments we were after. 

Following the keynote paper on Sunday night, a thr^-day c^iportunity to m€^ was cast into an 
oj^ratioral plan that was rei^r^ented by assi^ing each day to a type of teaming environment. On each of the 
three days, the plan was to look at four char^eristics of every exemf^ary mcxJei {resented that cteiy. On ^c:h 
day, for each model, we considered four topics: media; n^hods; mana^ment arxi measurement. Since eadi day 
represented a class of learnmg envircKiments, it w^ nef^sary to have a f^ n^dels presented on a day so as 
to represent both the breadth and focus of technological applications under the sf^ified oHiditions. The 
interestif^i outcome is the fact that on e^h day, or in other vtrcKds for each type of learning environment, the 
range of different mid successful technologies in use ts broad, arnl certainly no single technology stands out as 
thp hiKt one to use, either ovprall or in any particular learning envirc^n^nt, nor does it seem that any te^nology 
is automatically excluded from use in any of the learning envircximents. Sinc^ 3il of the te«^ir.wffOgies are possible 
within each type of learning environment, it seems that ctectsions regarding v^idi one to use must be based on 
other considerations. 
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The set of thit*e ttays was conceived not only as a continutim or evolutionary iiHxi^ei&sion, but also as one 
th^t was multi dimcnsiondl in nature. Two intuitively obvfous parameters were used to characteriia each day 
and to convey this deyetopmental sequer^e of days deating with envtronni€nts beginning with, but departing 
from, convention as the symposium moved from the first to the tfiird days. The first day was devoted to 
a consideration of wttat might be called Conventional lnstruction-*'sround zero*' tn the ^queiKe. 
durint)L lh«s day the partictpanis were tiusy fooRuig into the u$ti of technologies in support of conventional 
learning environments; we identified these as ''grouped and bounded/' The second day dealt ^ith ^jhdividuati^ed 
and bounded" learning environments. The third day itealt with "'personatired ami uf^KiutKied'* learning environ- 
ments. It became quite clear that technology can t^r a useful tool nq matter which type of environment is avail- 
able for teaming. The lower bound is simply that some technology can b» used in all learning environments. The 
upper bmimi problem, however, is more difftcult to specif: namely, what is the maximum a«? possible in each 
type of learning environment. At this time we do not have data to su^st what ^e uppar tKjund would be for 
each type of technoloc^ in each kind of learning environment. Relate is the optimization prctelem, what is the 
liest mix of existing technology to achieve a cost^ffective sotutt(Ki to the learning and satisfaction produced* 

Productivity. Profatems m6 a Propose 

The keynote paper presented Sunday night v^s commissioned f rcmi Professor Keith Lumsden, an 
economist from Stanford University, His paper was titled "Technological Innovation in a Hostile Environment: 
Problems of Increasing f^oduct ivtty in Hi#ier Education." Lumsden arises, from the perspectiv«» of an economist, 
that a number of oontrolHng conditions have had a subtle, but pervasive, effect on the operation of academia* 
Professor Lumsden identifies the main and related them to the prc^lem of iKtHiucing change in the 
practices that could tie accomplished at least in part, with technok^y or technological a«l which is available 
and could alter prcxluctivity in the academy. He also ar^es that the inrentive system, within the university 
for both students and faculty, requires a "massive research effort" it signifH:mt incre«es in educational 
productivity, regardless of how they are measured, are to be realized through the use of technological innovations. 
His paper is divided into four parts. The first discusses possible functions of the unh^erstty and (^oblems en- 
countered whenever differences in perceived function cause conflict. The second discusses major obstacles to 
change, The third indicates the inadecfuacies of much of the previously conducted re^arch o*i the economics of 
hi^rer education. Finally, he pfBposed a research effort to cxOTiine pedag^es while contelPit was kept constant* 
Each pedagogy would Providian alternative set of lining conditions while maintaining a ^t of educational 
Mijectives. The study should tie designed with the purpose of determining the relative costs ar^d effectiwness of 
the alternative peito^gies. As a general form of research to produce data in many areas this is s&en as a national 
f^tonty ; it is to provide data not n^w available re^nting student preference. 

' Professor Lumsden's paper discusses obsta:les to change and includes several intriguing notions about over- 
coming them. One that makes sound economic setne and an interesting psychoio^cal condition with re^^ect to 
the student and his opportunities to use learning resources is, in effect like providing the student with a book of 
tickets such as one gets to see Disneyland. The total can be spent anyway one Hk^. Within a uniform tuition 
systan this m>u\6 be possible based u;xh\ the known costs of a f :)ur year bachelor's exf^ience. StiKtents vtKiuid 
have to pay for their instruct kki in terms of the cost of the resources they want to u^ (the alternative pedagogies) 
to achieve a level of competence, self satisfaction, or both, in pursuit of their ^:ademic goals. Students who chose 
to learn something by means of an audio or video c^sette which might cost less than a human tutor or ever) a 
seminar, could spend less by making this choice and could then use the saving in other ways. 

In order to nrake this type of experiment possible a number of conditioni have to l^ met. The optiom 
would have to be available. This means thai uie instructional materials have to exist and be in alternative fomns for 
delivery or capable of being encotled in some alternative forms. Ideally, the alternative peda^ies shouki havp 
some established vaiidity-possibly data showing that there exists a student for whom each alternative has been 
found to produce the competencies invovl«i. Some may argue that this minimal condition is not sufficient, but 
that is anoth*»f matter. The administration of the institution involved has to make the decision to allocate the 
resources needed to make a set of options available. Data should exist for the student to permit him (or her) to 
process it tn dcadf^ on one of the alternatives. In order to do this research has to be done to produce the data Bad 
the siudern has to have had prior e^Jucational opportumties of a similar sort by means of which he {or she) would 
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be fKe|>dftHl to irwkf the type of ctecision required. A set of previous research prqjects has to be completed whjdi 
relate to the points mentioned and which included repr esentative samples so the range of hum«in variation which 
is Mkely to be involved witi have been studied. 

Fundamental to iumsilt n*^ firoposal k the assumption that the student (a cunsurnerl will get what he (she) 
pavs for, and thai the more one pays, the greatcv^ the cost to the tmtitution to develop, or provide what you get. 
This means careful economic analyses are neiKied so thf» pricing of alternatives is realistic. 

In onter to ^t the data that do not exist and to s^t the ultimate projiect fund^, Lomsden argues that ttw? 
research ^outd be supported by the Federal Government. His position is that the research is a public good and no 
group could benefit from doing the studies in an unselfish way. Data are needed on costs, prpb^ility of successful 
Gompleticm of the objectives for different types of students. Lumpen assumes the fiosition that different kimis of 
students will respond differently, andfor simply prefer one peda^igy to another. While the evidence for this type 
of thinking does not exist tn the published r^arch to the ite^K we vwjuf d like, it seems that there? are data 
which do suggest the iJN^itential validity of the "interMticm hyfmthesis" along the lines of the **idiographic model*' 
for C/^ *, ;or example. The lii^oblem is to ideotif y the charter istics of students tfiat shouW be known so th« data 
can be entpfoyed in making differential elisions both initially with respect to the teaching program used, and 
€turmg the course of learning as has been m<Kieled fm adapttt^ CAL 

Oie question raised by Lumsdem's paper vyas the strategy and criteria to be used in making the decision to 
conduct the study along the general fines su^sted, Lumsden sug^ts that we wait until reliable data exist on 
both cost and effectiveness, Ttw course of history ifcjes not ar^e that tWs has to t>e the basis for taking the next 
step. The best next step can be taken and if reasonably successful it can attract the necessary funding. This 
was what h^pened in aviation once the first hedvier-than air craft flew. Once the possibility was actuaHy demon- 
strated, then interest in the problem developed and tfte scientific basis few further developmwt takes place. 
Involved would be the development of the instruments and metNxte for determining cos^ and benefits. One 
could argue iSiat tiic requirement for the existerure of reliable cost benefit data as a precondition to the largB 
scale retiearch is a strategy that turns out to be overly conservative. The reason is tftat it could throttle down 
efforts to get the v>b doue in the name of rationality, while producing a negative social benefit. 

The whole question of research and especially educational research was raised by tumsrien's paper. Most of 
the particii:^nts dearly have been persuaded of its value, but they were not persuaded that educational research 
is sufficiently accepted by the academic community as a whole. Research on instrtiction does n<^ seem to add the 
legitimacy to the result that is comparable to the value ^ed by research in the «>cial sciences, let alone the 
physical, the natural scienras, or medicine. Here is aix>ther problem lurking in the background that imping upw 
the 2Ccompltshf7Tent of the effort su^ested by Lumsden. It seems that it is fwce^ary to get basic educattutk*! 
research itself honored and appreciated. Even scientists who in their rwn suu»i<;fmive ar^ ' Physics) are aw- 
vinmj of the value of research shift standards when they consider the procedures for solvifi^^f ofc^ms relatirtg to 
the teaching of their own science. This is a serious problem. There is a real "credibility ^p" for tJw faculty. The 
students, on the other hand, seem quite willing to accept and use educational research data when it is provided 
them. But another question complicates the situation. This is the questicm of priority with respect to our needs 
at this time. Lumsden seems to assume that we have the materials for the major experimental effort he prCHK>sed. 
Many others feel that *fife do not, they assert that we should be doing research on production arKl not on testify 
alternatives. To what extent the belief that there is an insufficient amount of material available is related to the 
point previously made about the credability of the data that exists concerning materials is not known. These 
l»?rceptions and l>eliefs are obviously related, however. 

In order to obtain critical data such as student preferences and the cost o1 tht> alternatives for achieving a 
particular objective, a sizable research effort is clearly neerted. Just how targe a project shtxild be mounted to 
answer the questions raised by Professor Lumsden remains to be ctetermined. That question and the nature of 
the research design to be used m the study should be the purpose of a research effort that is the initial phase 
of a longer-range program in educational technology. 
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Let us ind*cattj some of the more obvious a^nfcts of sudi a study. It would tiave to be planned on a 
national scale to include the Viiriety of educational elements representaiive of the different pedagogies, learning 
environmems, physical facilities, faculties and different student populations. The research ifesi^, collection and 
analysts of data and the coordination requires a centralized effort so the program would liest be manage by 
one qtoap, but, of necessity^ it would ifwotve many cofteges and untversf ties. 

Also ret|uire<i is a proia:t designed to ctevelop a list of the ccitical parameters tor use in a national survey qf 
educational programs and institutions. To begin with, a preiiminafy list would have to be deriml from m analy- 
y tical study of available researdi and from theory* The process by means of which the instr^iment used in making 
the survey if developwl, must, of necessity, iterative so it provides the required infonnatfon by successive 
approximations. This means the Study will extend over a period of years. It also has to be a rather comprehensive 
study ^nco information is needed about the cnarpcteristics of the students as w^l as their ^hievement levels fol- 
lowing dif ftfient learning experiences under different conditions, 
ft 

Unhke f irev40us large scale studies, the desit|.i vwtild te ^crated at least in part, |>y the hypottmis that 
then? are statistical interaction effects mher than that there is a linear correlatirm among the variables. Assumptions 
about thi? structure of the data base to be assembled are criticaf ^terminers of the ctesign us^ in cotfming the 
<lata to l)e anHly^<?d. If it is assumed that there are statistical inter^tions between variables refw^e^ttng student 
charact**rKncs* an the one hand, and variatiles representing the learning environmwt, on the other hand, then the 
instrun?ents used as well as the sampling, and the collection of the data shmjfd take these hypotheses into ^ount. 

Evidence of statistical interaction effects lietween variables representing students, on the one hand, »id 
Histructicinal programs, conditicms and learnimi environments, on the ottter hand, are nee<teti for differential 
diagnosis ami selective prescription of instructional exix»riences. Since tfre number of variables to be consicteied 
IS potentially very large and the amount of existent information retatively smati, careful planning of research is 
indicated Large samples of stuctents are required. Selected socioiogicaf, anthropoitgicat and psydiological data, 
fm example, would have to collected fi>r ♦^^h stmh^nt. Each variable repussentlng a student char£K:ieristfC would 
lie related to one, or more de^ndent varialiles revealing to various kinds of student performance, to rate of 
learning, to achievement level and to various measures in th*? affective domain. In order to answer the cost-benefit 
questions, it wouUf also be necessary to include data relating to the costs incurred in achieving learning effects and 
the dfffert^nt cfffective reactions. 

To ot)tain all of the kinds of data rec|uired, it is clpr,r rhat some new instruments and procedures would have 
to \Hf developed. For example, colleges and universities tio not routinely collect data relating to such stutlent char 
acterjstics as thf^ir iwsondltiy characteristics and level of anxiety. Standardized tests of student adiievement are 
not usually qiven t>oth at the teginning and end of each course or every ^ademic year, yet this is necessary for the 
measurement of learnmg ijains. Collets and universities often do not qiye final examinations but even these are 
qiven more often than arc ^>re tests. Information about what students already know is critical for the assessfoent 
of what they have learned, and for making instrtctionaJ decisiorts atx>ut what to terch individi^l students. 
The lartie unanswenHi (question about the efficiency of academic programs is how much time is spent by the 
faculty teachinii thmiis stu{lt.»nts already know?" The only way to answer this qi^stion is to collect information 
before the studi»nts art* prosenteti with a new learning exp^ience. The n^chanics of doing this can vary. For exam- 
pic. the data for the rH?xt course can be collected at the end of the last learningexf^rience at the same time as the 
final, or post-test, isijiven. The one testing experience can t)€ both a pre test and a p^*-t«t. However, such 
practices have not yet l>ecome a regular part of college or university plan. As a result, it is unlikely that the 
crititMl data needed to makt? the survey of existing data for use in planning more definitive studies of alternative 
pedayofiies wilf !>e avdilaf)fi?, Therf^fw-e, while the need for a survey exists at this time, it should be planned with 
this kfiuwiedge in mmri so as to maximize its payoff. To do this Its planning should include the collection of 
data from the administratfon and the faculty to determine their willingness to chanr^? their practices so as to 
collect the required lUu. It atso shoiild include questions dealing with the ability of the institution to meet these 
recfiiifements. This part of thi? study should reveal possible deterrents to the OTilection of sudi data so they can be 
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Co4t tUjta iU> not tixjst in coHeges and uniwrsities in the ttetaHtnl form required for cost-benefit analysis of 
instruction. Most colleges and universities do not tjud^t or analyze the cost of individual courses of instruction or 
eveti urograms of instruction. The cost unit is usually the tk'twrtment. and a depaftmef»t may deliver instruction to 
a variety of different programs, therefore, its costs have to be analyzed in detail. With the present aGcoutiting systems 
m colletjHs mi\ umvetsitnjs .t would h.- d,ff it u«. if possihl,- at all. to specify the costs of even the more obvious 
Items such as the physical plant improvements and now construction. ThesaT facilities frequently are shared in their 
USB by a luimUi'r of different programs. The amortization of their costs and the distribution their costs are probfems 
that must iw itealt with in a comprehensive economic rese-^rch project aimed at the problems of cost-benefit analy- 
sis wiihi4» higher e<lucation. Space, supplies, materials and persoimef costs, all have to be identified at the level of 
the workinii unit for which a cost-benefit analysis is Jwing matte. AH of the contributing components have to be 
mdmltjd m the analysis no matter what group provides the service or the facility. The complexity of the problem 
and US relative novelty intltcate that there is a significant research and development requirement to produce the 
models and methods needed to permit institutions of higher etlucation to particpate in a realistic way in a cost- 
iKjnef it analysis of their educational efforts. 

Th.) economic mwlels and the accounting and Initlgeting procedures required for a cost-benefits analysis also 
hav»! to be developetl. In fact, this is a necessary first step, but one that could run concurrently with the research 
and development work involved in generating tfie student perfonnance data required to determine the benefits 
{«.«>.. If.irning gains, satisfaction) from particular programs. Only if similar accounting procedures are used by 
different institutions can ^ multi institutional research study deal with the question of cost-effectiveness m any 
meaningful and satisfying way. 

It IS c(»?ar that we can expect differei^ces in the basic costs relating to education simply because of differences 
relating to the |Mrt of the country in which schooling is provided. Salary and materials costs vary over the country 
and taxes also show great variations. Tuition varies for the fuivate and public institutions and the newer ones are 
amortizing buililing ami other facilities acquired at much greater cost than the older and more established 
institutions. 

From this bnef analysis of some of the many problems that confront a cost -benefit analysis, it seems that a 
definitive answer is not likely to be forthcoming in the very near future. The point that needs lo be made at this 
rime, it seems, is that the education industry is being plarad on the alert; it has to bewmw costcmscious and 
develop mettiorls and procedures for good cost accounting. The cost-benefits analysis is a one edged sword; it only 
cuts <jut and dfwvn. It is cspeciallv sharp in relation to etiucational technology, and yet labtw intensity is so 
characteristic of etfocatinn that it is hard to believe tJiat technology can not, over the long haul, prwiiu^ saving. 
We new! lo ask why education is the last stronghold of the cottage industry. We also need to ask whether we can 
continue to afford it as a cottage ifflHustry. 

If we intycmni to exdmtne this very important set of problems in a serious and methodical way, in the manner 
It clearly deserves, there are a number of positive th.,igs that can be done. Data of the kind su^sted in this 
seminar cfMild be collecteil ijoth from the input and the output side. Mottels could be ttevelop«f. Simulatiwis pre- 
(wretl and run. Empirical studies could l>e conducteri to test the simulations and to validate their predictions. The 
information from the emfiir ical studies could then be used to imf^ove the models. The models could provide the 
twsi*. for redesigning operating sy<items. 

At this point we see .i critical- need for a set of organized research and development activities which could be 
dofivod from thf> papers presentetl and the growing literature in the field. Its form would embrace the study 
dcscrilH'd by Ltimsden with the additional implications included along the lines just ctescribed here and later on. 
Each project, however, would have to be develope<l and relateii to the whole set so as to provi(te a prcgfam of 
research all of which should be conducted within a systems framework. Not only its mana^ment, but also its 
continue<i review and systematic modification should t)e centralized to achieve maximum efficiency. This program 
of research ami Hevelopnrwnt cuuld be the defining activity of ISIIE in the next several years. The collection of data 
on J national basis to provide a data base for the wide lani^ of educational decisions to be made is an important 
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ndtional need. In f&c% n is a critical need at this time. Educational t^notogy, as said before, is making this need 
apiJdfent and. in addition it is castii^ rtie problem into an ciperationat form that makes the appro^hes to its 
solution clearer. Technolc^^ especialtv comimter. technology* could an impor tahl tool to get the job done. 
N|E cmtUl consitfer a division within its administrative aeyis that operates like the Bureau of Labor Statistics, for 
example By starting with that model in mind, but at a lower level of support^ this division of NfE could provide 
higher eduGif ion with many very t>adly needed services. 

Professor Lumsden's paper and some of the tmplfcations drawn from it mi^t raise the spector of standardi^a- 
tion/control and restriction of educational practices in the mir^is of many educators, tt seems to tfie that this is 
clearly unfounded, and appwrs to be a »t of fear^ which reflect lack of knowledge and faith in the knowfec^ of 
individual differences among people and institutions, not to mention ethnic and sub^uttural groups. The psycho^ 
logical variations amonq students are cp«^ainly as complex as the variations in physical st«e, preferences f<» food, 
ctothiftg. htHisjng and att the other physical artifacts of our society. Why then shoutd It not be the case that 
Mariaty will also prevail in education, tn fact, it wouW seem reasonable to assume tiiat the variety mi#it ewn be 
greater in its range of variation than it is for any of the physical characteristics* 

When we consider the (Question of c^timization, are tf»5?^king not only of the most efficfcnt learning en- 
vironment for an individual, given his. or her, peculiarities, twjt also his, or her^ i:^eferam^ which includes 
attitudinal data relating to the preferences the stuctent might have for a particular type of experience. Also, if 
accept Lumsden's recomrrmndation then different individuals will probably pay more for certain experiences and 
tess for other educational experience just a^ they do for the different antertainn^ent, food or other consumer 
goods, options. Even in what appear to be more hi^ity controlled economic units stKrh as manufacturing of 
competitive consunmer products, a wkie v. riety of contpanies lorgani^ationai systems) and production plans co- 
exist in a free ecoftomy. There is little to !aiggest tfiat the eduoancmat industry wtll move to d ccmsistent 
p^rterri or anything approaching uniformity as more and better tfiformation becomes available to guide decision 
making and as the technology imprcwes. The proUern will more likely be one of ^tting the industry to XBe the 
available data at all let alone in interesting ways. 

We n>ay have to move in education the way the country did in agriculture. We may have to establish, at 
various campuses, or centers, the counterpart of the agricultural experimental station. These ^its mi^t be called 
Ediicationai £xperimmt^ Stations. They would have profe^ionai expertise in technology and the information 
that could be applied in efforts to improve the diff^ent institutions and help i em make use of available informa- 
tion to solve the problems indigenous \o their instructional and educational obiecttve^, thorr student body, faculty, 
alumni and local conditions. 

Through a national r^twork of SiAjcatfmaf Experimental Static the use of tedinotogy mig^it become more 
widespread and the available information more efficiently utilised. Mc^ls for sudi activities are needed. There is 
every reason to assume thai if interesting and viable models were developed tfwy would serve more than ji^t the 
institutions of hi#>er educaticHi. However, tjy starting with these institutions the investment in their devek^ment 
would be less and the payoff neater. The diffusicRi of their effect throughout the country could tje facilitate by 
university and professional societies. Furthermc»re, the limited personnel available for such an undertaking, at this 
time, could be concentrated, if there is a smaller number of centers. This would permit the assembly of a ''critical 
mass" of personnel sufficient to get the job done. 

No institutions of higher education are currently training the type> of personnel need^ to provide the 
varieties of expertise needed for the staff of an Educational Experimental Station. Some institutions are beginning 
to,train some of the specialties that would i>c needed in a team effort. There is a curricular problem. The field 
ne*v<s a comprehensive program, especially the graudate training of professional educational tecNioioglsts. It not 
only needs specialists, but also ^neralists with the mixture of knowledge and skills required to deal with many of 
the complex problems tn education which are not now being faced by properly trainKi personnel, nor are questions 
being dealt with by research and development in a scientific manrwr. They are being treated as philosophical and 
ffitinttvtt issuer. 
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RECOMMENDATIONS 



Professor Lumsden's lirgument that public funding is essential i% sound mii convincing. Fur this reason 
recommend that NIE dei^lop a program in educational t^noic^ with the foMowing areas: 

ftecommemistion 1: Study yi ants be provided to develop one, or more, pkms, with cost estimates and time 
proiwtions so as to n«ke explicit the dimensions of the "massive'' re«?an*i effort propped by Professor Lumsden. 

The study grants w«iuld have to tfeal witt^ a v&t of i^fated questions. One is thi* question of feasibility. This 
would entail a survey of the available materials thai could meet the lequirement of having alternative p^fagogies 
available from which students could sdect the one they preferred. The i\uvHtm of diffeient alternatives that is 
made available to a group of students to produce preference data also is important. One reason is that students 
have to fearn how to make the decisions retired by this wnditlon and to do this they to haw the opportunity 
to learn what the alternatives mean to them. The cost and effectiveness of the alternative pedagc^ies with different 
kinite of students also has to be determined. Sampling is an important consideration since students from a variety 
of b^kgrounds and studying in a variety pf institutions eadi of which provided them with a set of options would 
be needed. To collect the required dato cm student preferences for pedago^es as suggest^ necessita^ a "free" 
economy, one in wttich the alternatives can compete with each other for students in a particular college or univei^lty 
setting. The research design should not only allt^v students to l^rn with i^agogies of their choice but also should 
assign some students to nwpreferr^ peda^les. This would make it fK>ssible to determine what effect, if any. 
preferential notching has upon the outcomes f^ co^itive, affective, and skill learning. &jme learning moctels for 
example, would predict that not having the use of the preferred pedagogy would produce tess learning because of 
reduced motivaticm. 

Ftecc^nmendation 2: Development projects ^oukl be funded to design models for measuring input and output 
and to evaluate measuring instruments and r^<M:edures for education^ systems using e^h model. 

Models as well as appropriate meaufuring instruments and procedures are needed to provide a rationale for the 
development of instruments and lo validate diem. The measuring instruments and procedi^es would have to be 
devrioped for botfi input and output \x^tm\^&n and should cover a variety of indices. 

Hecontwendation 3: Development projects should be f uncted to create and test irnxfels of student and f<icuity 
incentive and reward systems based upon both r^ychc4ogi(^ ami »::^nmm: data and tfieory. 

These models and their implications would be based upon available data and theory. They would be desired to 
permit an instituticm of hic^er education to make available p^agogkat alternative In ways that allow the alternatives 
to com^te with one another. They also would reveal the critkral conditions for empiricai study. 

Recommendapon 4: Research projects shcHjId be funded to collect data on student preferences for petbgogical 
alternatives and to study statistical interactiw effects between lor among) student characteristics and peda^ies. 

Tfie research would have to be designed to look at the question of f^eferernies in many, but selected, institu- 
tions. The pedagc^es would be related to the dimensions of a research desjgn, and the additions would have to 
provide the needed mif?asures and allow for the analysis. The results of the projects established under the first thri>e 
recommendations should be related to the planning and conduct of this research. 

Grouped and Bouncted Learning Environments 

A visit to a collect or university campus reveals that *he academic rooms a!n-^.)sf always are used for a lecture, 
less often for a seminar or a laboratory demonstration or expei iment. It is m thii sense that these s|?acos cdn be 
said to hcuise grouped and t>ounded (in both length of time and in scheduled time, as well as in topic) learning 
environments. They are referred to here as thfj'^'conventional/' or traditional, ones used in higher education l>ecause 
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ttwy domifute the scene totlay ure Ueinq iniitt fur use in the etlucatioiiiil institutions af tomurrowr fn Ihem 
sludwis •*tdM#iT'* as a yroup, anrf the instruction takes pface in a 5p«^i!ic number of predetemiined slots of 
time for all who enroll. The students are exiwteil to enter the rocwn at one time, to take a course together, antl 
til tie minimalty resiKJiibive and n-atrtrvt* tu what ijnt >s <Mt, Beirni •^t<M^thf?r" seldom rneans that the students interact 
with each other in the classroom to either fcUs^m oi learn, Tfie set of conditions characterwring Grouped and 
BoutHied Learntnci Environments coit tie representml as a fwnt on a continuum that also contains the other two 
learning envriomnents mduileil in the symposium. 

One of the most widely discussed and frequently adopter) chants in programs of instruction is the time 
allowed- Carroll's learning model equates time with aptitude tor the tvp« of learnini} involved. Many schools are 
shdttng to a three year rathtsr than a tinir-v<L»ar curriculum hn a iiachelor's degree. But, *is is not a manipulation 
of time tn the sense that Or. Carroll uses it as equivalent to aptitude. In Metiicine, where competencies are a 
i:Kitndty concern, the changes in curriculums consist more in time chants and in ^ifts in ^u^nce than they do 
in p<fdaqogy cham^ However, while competency is a more valid benchmark than the time available for learning 
in an academic program, (earning envirwiments have tended to be bounded by tim-. This has the signif iGam effect 
of makmg aptitude the determiner of successf j| completion if other factors are equal and if we ^cept Dr- 
Carrotrs theory. If we liken education to the process of aging wmes, then simply k<?eping the student in the 
coMegtaie Inittle for the rit^t lt?ngth of time is all thai is necessary to produtx* a f ini^d product, according to 
ihese simplistic manipulations of the time allowetf. The processes m making wine go on in a closed and statjle 
envwonment ami ilepend more on nature of the raw materials put into the environment than upon the charac- 
teristics of the environment. With students in a learning environment the situation is different. We do not think that 
most acattemics really l>elM?vt? in the wine model for students; hovw'/er, many critics apjMfar to act as if they do. 
Many administrators also i>^t trapped as indicated by their prcttitection to acijust programs in terms of time rattier 
ttran in terms of what the student is to learn to do . Time should be a tiepemleiit variatrfe, a result derived frwp an 
academic plan, and not an imlepcndent variable. 

Alternative fiedagogtes are the means by whidi a tixetl s<»t of oljjective^ Mating to knwvledge, skills, and 
values are to be achieve<l l>y the educational system. P**<!a^ies that an» truly alternatives are designed to develop 
the same competencies. While admittedly it is much more ilifficutt to enumerate ctrnifMtencies, than to adjust the 
time allowed for study, and even more difficult to get academic and professional groups to agree pn ttiose competen 
cies which they feel are a minimal set for a person completing a degree, Jt is neverttieless true ttiat that is the task 
which the academic community should be working on. It is proliaWy necessary to work in sta?^ until it has 
3Chieve<f an acceptable approxinnation of what the group would accept since this is not an estaWished and accepted 
way of d4?finin<j their task. 

Toward the end of ttof inmg educational objectives, whether it is for conventional instruction or more person- 
alized environments, thea' is an evolving eifucational technology. It is a development of the fifties and sixties, 
ftelative to the age of the university system it is very young, but it seems to tn? growing rapidly. This is a technology 
that involves ideas and methodology, not just harr^are. To some extent its growth has been stimulated by supfiort 
from the fari|cr external environment m w^ich we live. This the clamoring for an accountable educational system. 
You havo to know ^'ha: it is you (*n\n^cX to produce by the system in order to ^lold it accountable. If there are no 
agreed uron c^jectivr s, then anything that is fwotJuced has to be accepted as a useful outcome, Tho^ of us who 
have wurke<1 with te ichers and professors to ctevrlop statements of obH?t::tives for the fHjrf>ose of cfef Ining the focus 
of an educational pr >gr3m kiH>w fuM well how difficult that task is and how much it is resisted in its accomplish 
ment by those more jsed to other, older ways allowing them more freedom and the illusion of success no matter 
what they do. Sor.c of ^his n^sistance may Iw due to the technique and the technology itself which is used by the 
programs attempting to accomplish a technological solution to their problems. Umloubtetlly their techniqi^s can 
Ix! improved upon, but m addition, there seems to Ix? an inherent f)roblem in the technology of developing objec 
tives which is ridficult for a faculty to do. This fact raises (questions. One of them is the f|uestion of how to 
identify olijectives. Another is how to tjet agreement on obfectives by the faculty involved. Any rifsrarch plan using 
alternative pedagogies depends cr ihcalty on the use of a smqie set of ol)iectives for .ill of the alternatives. 
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Pr<^tiiy relate to the pr€*Jem of getting agreement is the f^t that one of the most difficult ccmcepts to 
get acr«s to faculty 15 that the list of objectives is a ^^if icatkm of ttie nmknum tevel that many stuci^ts mtt 
achieve just as it^y cfci new. Qm result of better estaWi^iect and orgamxed leamii^ prcKJti«:ed trough the use of 
technolofty and the time saved for the teamer with high aptitucte is the €HH>ornirtity to ^ieve even miore. 

The reports we have heard (ndtcate that in spite of the diff icuUies, it Is possible to get ayeement on objec- 
tives and to ttevi^CNP «ef s of them that are ^cept^ie to p^oups. Tte axomfili^nient of this step often is ttoder- 
rated in its contribution to tl^ entire effort. It is a step whidi «^en ^onptishod in a non trivial way restdts in a 
^eat de^ of re-thinking of courses and curricuJa typically with greater sensitivity to the psyctKjfosNsf , typtcaHy 
the learning problems of the stuifents. but also to tfie organizational i^iestions and the priorities involve in pre- 
senting information, concepts and f^inciples to students. It f% interesting to relate this stt^ of getting agreement on 
objectives to what h wpens conventionally. In the conventicmaf situation agreement cm objiectives is ach^ved "after 
the fact*' rather than ''before the fact". In this older mode of doing busings, the book is written and published. 
Then agreement* at feast to an extent, h achieved other Insin^rs dioi»e it as their text. 

It is apparent that sigpiificwt innovations can and are takii^ptaK^ within tt^ grcHiped and bounded ieamir^ 
environments. Where this h the case, the institution^ in gena^at, carries on business as usual in most otf^r areas 
and it ^commodates the changeii, Stice and HerefcH^d teport this initial f^e of their efforts at Austin as did 
Hurst for Purdue and Miller for the many institutions using technolc^cal aids. Not all of the tedinofogies have 
the same potenpal impact on the imtituticm, nor do they alt serve as catalyst im dian^ or as instruments in their 
own ri^t, but most do. It is imp«rt»it to consider the implications of the change a^nt ft^ion of the technoto^ 
gies. Also this functicm should be considered in rdation toeach ir^titufic^'s tc^g range plans for ctiange. sirux 
some technologies vvHI be mo c acceptable initially than others and since €»ch does result in review and deep 
analysts of instruction in both its contmt and process dimensi<ms if it is done correctly and serioudy, Cknx started, 
change do tafce place in the way the fatuity and the institution do tosiness^ in the attittKte toward teeing and 
in the ic^g-range planning of curricula to keep it €om»s^t with duinges in areas of study, particularly in the 
professions. In this seme technology is not Mt »t episodk: dtan^ a^t, it is <me that inteyatos changes ami blends 
ttem into a ISNr^r more coherent cfmceptitxi wifli its own (nitential f^w growlfi and c^ttqsment. ThriKi^ the 
use of many of the technologies thems^hfe^ the or^i^ing wd evolving ^ocess itself can be facilitated. Whtdi 
t^rhnology is used first and vA^at tedinoiogiesare used in particular combinations seem to be questions wlitch at 
this point in time, are less critical than the simple (j^festitms about the process of getting started and of gaining some 
experience in the use of one or nwrc of them. A pluralism of technologies is i^rtahily indicated for larger se^nents 
of instruction such ^ courses. Options should not be dosed cait, fH^ionty is to be ^ven to chan^ for ttie Improve- 
ment of learning not to a parikruiar technology just because it is inexpensive or simple. 

Professor Arthur Luehrman, from Dartmouth Uni^rsity^ correctly takes the position that we shouk) not 
expect thafcommiters wiH only redix:e the manfKivi^r requir^nents for manajpng instruction (CM!) and for 
delivering instruction ICAI). VWtile he acknowledges tfwse 0}aH and their value to the educational industry^ he 
(^ickty poin& «Hit a much more profound effect of the con^ter on stuctents whii:^ education must recognise 
and make maximum use of. This unique ccNitritnitkm of ccnnputers is the way in which the stuctent's knowtedge of 
computing can affect the content and intellectual structure of his whole education and determine in proiounrt ways 
the manner and mode witii which he relaies to {Koblems. Luehrman's thesis is "...that computing is a new wd 
fundamental resource in the same sense that reading^ writing and mathematiis are furKlamental refsources," With 
respect to the way institutions relate to con^uters Luehrman prcposn; that the model be the university library. 
The library is a "cost center/' not a "prof it-^-loss center/' It is justified on the value of the service it provides. 
Using this mottel, Dartmouth provides computing services to all-faculty and students alike-- and oi>etaies its 
tinte-sharing system (DTSS) wth Basic as its programming lan^f^. Not only does it serve the cam^s, but also 
thirty other institutions. Of ti^f 200, oi more, terminals that can »xess the ^^i^f^n\ irm\ the campus, about 
can be processed simultaneoudy. 80th funded ^d unfunded users are served and !hf; Jwlfs of the unfur^ted accounts 
are |:^id for in the end by transferring general funds of the college into them. Thf> total computer center budget is 
srtKwit $1.2 million for Dartmouth usage. With 4,(XX) students, tfw? average cost is about S^M). per student year, 
or approximately 9% of their annual tuition fee. 
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Pf o^t COMPUT IS an effort to deveic^ anc* distribute coufise malsfials dfiating with computer use. It «s 
ill Diirimotith. furuted by NSF and avatfaWe ta faculty membef s at any mstitultan. Jt$ goal is ta publish and 
d<s*,fm»nate approximatefy 20texttK>okSf^ monogr«^hs. Awards are made to mdividuals* whose publication felaft* 
m ?M»w and <nteffstinij uses of the computer m etJitcaf ion. This pxoivcx «s tlesi^ed !u fiH tne gap or shortji^' ot 
0ooil materials far use in courses where the use of a computer makes sense. 

Reamintemiatii^ 5: That funds be provided ip selected university centers meeting a set of criteria with 
mpect to educational technology and scholarship. Each woutd serve as an Educational Ex^Myr imental Station. 
And among its functions it would support projects like COMPUT, This would mean both supf lort tu the authors 
of texts that use computers, of CAI materials and of CMt programs and assistance in making these matenats and 
programs useful to the educational industry. 

h^ofessor Robert Hursl. Department of Blologicai &:<enc«5 at Purdue Unrv irsity, described ^ very 
succr?ssful use of technology, the Audio- Tutorial system of instruction lA-TK It too is a child of the sixties, a 
tee«tai|ed technology. &-*|imiing with the caveat that you cannot legislate chan^ in Education and get a viable 
effort. Hurst r|uotes the (^megie Commission's Fourth ffevtrfuf/ow on the criteria that should be used to detei 

mine whether technology should be employed in an instructional program " the learntng task to be performed 

should tie esTsential to the course of instruction, and„.the task sh<H4ld be performed better with the use of techno 
fogy /" He reports that Dr. Postlethwait te^n the A T system at Purdue out of frustration wttfi the lecture 
methiHis for hetero^jeneous groups of students, therefore it is important to distin^ish between a lecture on tape 
and u tutorial on tape. Tlie latter is a concept which itself is a technological development. The tapes ate pte^ 
paretl using a plan that \m]ms with a task analysis and moves from it to objectives. This set of steps is common 
to airnosf all of the t<?chnologies, tn addition, the tapes rec^ire the sttKtent to do thin^. Audio tapes were used 
t>ecatise they were available, durable and inexpensive to produce and use refiabfy. Professor Hurst reports that 
A T IS a "now" technology. Critical to the tlesi^ and use of A-T is the procedure by which the tape leads stu* 
dents, loifically from one experience to another. They systematicaily ^t the student to the desired performance 
level as specified in statements of obiectives. The tape serves ^ tfte integrating vehicle. The presentation is con- 
v€f satmnal and pacing is individuaiued allowing student to stop, hack up, or to repfay, as nec^jssary . Also 
important r< the fact that A T frees the hands and eyes to ioteyate with other experiences. In practice A T is 
not always audio tape, 8 mm film is also usetl In 1969. the concept of learning for mastery was introduceil, 
fearninq time is ncvv the depenctent variable. Students take the time they t^quirf* to achiew the specified minimal 
frvel of mastery. MiniH:ourses are the basic element of the system. "Hiese "little" courses, each with a beginmng 
anil imding, arc designed around a single topic. Upon completion of a mini course the student is given feedback 
through reports of his iwrformance on an oral and written pui^. 

Since the studefits learn for mastery, all redundancy can be eliminated. Experience at Purdue resulted m 
thi? f^hminatfon of 25% of the redundant material that had t^en in two conventional courses. This illustrates the 
roah/ation of the Carnegie Commission's "Less time-more Options," 

On? of tlU' problems with A T is nmiageffiefft and experience indicates that the approach could be l>enef itccJ 
substantially by a CMI capdbility. 

Recommendation 6: That a research project Ije funded to develop a CMI system and machine indepei^ckvit 
progi jfn canatile of haftdlmg A T mun courses, the tests and feedti^k to stuctents. 

Professor David Miller's presentation include! a definition of ediK:ational technology; namely, that of 
Norman MacKin/ie -"the systematic study of the means whereby educational ends are thieved." The mam 
thrust of Professor Miner's paper is to pi esent the wide range of low-cost technologies now avatiat)le for use m 
fiigher education. Consistent with Professor Hurst, he proposes three criteria that determine the spread of usayi»: 
( 1 > siitjsfy an educational need as seen by the instructor; (2| readily available and convenient to use, (3) relatively 
low m cost to student, instructor and institution. 
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ProfesscM' Milter hds deveiuiicd 3 "lar class" an "individiwl study !r:ocfcl'' whidi can aa:ommod3t«^ ail 
of the low-cosi tedinoiogv. Each model offers a way to increase itie stucknt-to-facuity ratio- The fmt uses m 
rncreased cfass size. The order bf priority of three technot(^e$ examined for manner of use, cost, ccn^ienc^ and 
availability are: (tl the overhead projector; {2) sfictes and (31 film. His seccnnd model covers the variety of 
mdividuaf uses of technology ''the dhalien^ of desic^in 1 uccessf ut systems that provide individualized opportunities * 
for fitted self-study depend heavily 00 lew-Cost people and, when appropriate and available. I«w<05t technolt^y/" 

Recommeiuhtiim 7; That the higher education community's r^eds would be best met if an infont^tion ser- 
vice and evaluation center were funcfed. It c»uld provide date, toth technical spscificaticms i^d cwt and also serve 
as a clearing house for experience data provided by center studies wd «ser reports. Institutions could be kept 
current on usa^e and on materials available ^ well as tecJinlque^ to i^oduce materiais for tow-cost ctevices. Ideally 
this function would be included in the set provkted by the Educational Experimental Station. 

Professor James Bess' paper ''Envtrcmmental Supports for New tnstriMzfional T«:hnorogies" covered an of 
study that is badly in need for research. He con<»ned himself with the itwasumnmt of effetrts of new or expert 
mental instruclionai technology. Not only is ^ere tfie obvious need to assess the changes In the learner as broadly 
as possible, but also in the environment in v^ich the learner lives and works to acquire knowled^, attitudes and 
skills. Bess be^ns with a discussicm of variaUes that mifi^t be used m stiniies of techncJc^y. tn colfecting tbta re- 
latmg to the environment it would be desiraUe to have information before and after the technology was intro- 
duced. The difference could provide a measure of change, and, if sensitive en(HJ#t, an index to ^e nature of the 
changes. Changes in faculty aie a potentially interesting, i»it difficuit set of v^ata to %c^ire. 

ftofessor Be^ deals with interpersonal environment of .the campus and examines variablas relating to the 
impact of teaching learning technologies w students. He identifies conflicts between the r^ds of the students and 
the aims of the curricuhim, e.g., immedtato ^ future, subjective and in^^personal knowledge wd skills v$, 
pbiective vocationally related information. While the two sets are related they unfortuna^ly m often op(K>$ed to 
or>e another "...particularly m bounded, conventional systems of hi^er educatk»i, desi^iers of new technologies 
have not adequately comprehended the synergistic nature of institutional and student needs md have concentrated 
on the latter/' Bess argils for "A much more cimipr^nsive conceptualization of the envircmmental conditions 
requisite to the satisfax:tion of both af f^tii^ and cognitive learning.,./' While he recognizes that mastery of 
academic knc»^led^ and skills can contribute si^lf ic^tly to ones $elf*perceptions and feelings of worth, he ixyints 
out that the established pattern of the institution to concentrate on this is also the primary thrust of the educational 
technoiojes. To change this it is necessary for fwutty to recoipiize ^at the out-of-ctess activity of students has a 
direct relationship to the in-class foaming. 

The nature of the relationships am<my the technolc^, the environment arid student's growth and devei(^ 
ment should be e>rpHca^d. "Beyond the me^urement problems in evaluating the effects of new instructional tech- 
nologies on students are a whole host of other envirc»imentai factors particularly tht^ in^ving ^e faculty/' 
He summarizes by pointing out diat Jearninri whidi affects the wh<^ persc^ requires careful c^ign of the total 
environment in which the process takes pl^e. In tounded conventional systems, it is unlikely that new instructtcmal 
technologies will result in profound benefits yvidiout positive social system reinforcement/' Measurement of the 
attitudes in the environment in which new instructional technologies are introduced will reveal the presetKe, or 
absence, of values which support long^l^ting learning and which encourage attention to student growth and develop- 
ment. In our opinion this ar^ment supports the following: 

Recommendation 8: That research be supported to develop m^sures and experimental procedures for assessing 
the impact of different technologists on various instittitic^al elements, 

Recomm&^dation 9: That research and development of a fairly extensive nature funded to work on problems 
of social and attitudinal conditions in the environment surrounding technological developments - the ripple effects. 

Recommendation 10: That support be pr evicted for the development of n»ethodology and instruments as well as 
studies of affective and sociological effects of t^hnotogv. 
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Chdracterizations of the host environments m ways thi*i * t;veal their receptiveity or hostility and their abif ity 
to dccommodate the chai itj^s required to make mdividuaf izing terfinotogy work well, are an important national 
t*dMcalfun«it priority. Th*: tnttirpi»fsonal «Hvtronmt?nt of both faculty Jnnovators and student users is a srgnificaat 
unknown that requires sutisfantiat analysis and study* 

RecOfnffwndaiion 1 1: That institutional research be supports to (a) develq:i ways of using new instructional 
technologies to produce a better relaticH^ship between the stuctents' academic and non-academic life; and (b) 
support studies of the "ripple effects" of technologies within the instituti<m. 

Inditfidudt28d» Bounded teaming EniriKMinnents 

The individualized, Lounded learnmg c^vtrcmments repre^t the efforts, typically of one or two more 
enterprising, risk-taking faculty memt^rs, to change an institution's way of doing business. Tlie chan^ are a 
matter of degree* and there is a iarc^ gray area between things descrited the first day and those covered on the 
second. 

ft^ofessor Bunderson and his coilea^es at Bri^^iam Young University represent a jointng of forces in that he 
bfid some of his colleagues at the University of Texas moved to B.Y U, in 1972 to <^nwlidate the NSF funded 
TICCIT project with Rff ITRE Corpora tiofi. ^Is consolidatii^ fits what his been referred to as the "critical mass*^ 
requiremimt for ppr^nnnel in a significant educational technolc^ effort. The project effort is a team phenomenon 
involving a university and a not-for-profit coiporaticKi working with fecter^rf funds, MITRE pcrsonnnel ar« respon* 
sible for systems engineering and the B*Y<U..facuUy for the instructional desi^ Sf^if ications for the CAi system 
and the courseware. Professor Bunderson acknowledges the importance of the instituticnai and extra-institutional 
attitudinal and sociological problems but does not cteal with them. He also describes the immediate "'context*' 
or organisation at B*Y,U* in which this project is housed. 

Professor Bunderson deals with the "...disciplined ctesign approc^h to the solution of educational problems 
and the interdisciplinary cross-fertilization between the designers arid developers../' ffe pomts out a significant 
but not wictely recogni/ed point in this connection, namely, "It is the explicit reco^ition and use of human 
goals and values,... in a rigor ous« empiricdily based process of desi^ and tfevefof^nent that pi pvictes the critical 
distinction between a desi^ science and a natural science/' 

The paper deals wllh desi^ goals, institutional level goals, effectiveness goals for individual students, and 
c^als for educdt<^. This is an interesting toxonomy that could have generalizable value to the emerpng ^ucationa! 
technology. E\«n more significant in the long run the concepts and technioues described. This is one of the 
most developed presentations of educational technology as a process available. It reflects the concern and atten- 
tion to the empirical and conceptual base neected to produce a vigorous edt^ticKial technology. 

Recommendation 12: That funds be provided to develop a handbook describing and illustrating courseware 
developmen^techniques. 

RecommendatitHJ 13: That funds be provided to develop mana^ment \niofmaXior\, including role (job) 
descriptions for the personnel involved in instructional systems development and evaluation. 

Dr. Parry demonstrated the PLATO IV system and described its operation. The system is one of a kind and 
is not a production item in the inventory of any company. It uses components such as Control Data Corporation 
Cyher-73 computer, an Owens-tUinois plasma display panel, Ma^avox terminals and computer 4;ontrolled 
a:ctfSSories ctevelo^d at the ijhiversity of tllmots (randimi access ima^ selector, mdio device, and touch sensi- 
tive surface on the screen). Remote communication has to fae over either microwave or cable TV to meet projected 
cost figures, but the system can communicate to remote users, as was the case for the demonstration at the 
symposium, by normal telephone circuitry. 
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Lessomior use cm FLATO are wricten by faculty fii the TUl OR author !an^^ which cmty exists on that 
systetn but which is api^n^ntiy easfiy h?arne<i 

This IS «i Unf* mmputei iitifity tirthcatcd to instruction. Its ttesign was tiased upcm an effort to minimize the 
cost of tktivery of instruction. WhMt* the profecteii costs figures have nut be€^ met, the system is operatimiat and 
impressive in its capabihties and in tht^ quahty of the delivery. 

The primary recammendiitfon following from this report has alre^y been activated* NSF has already funtted 
an evatuation project at ETS for both PLATO and TICCf T. as indicated in the paper by Or. Anastasto* 

Stnce and Hereford's presentation covers what mi|^t be calted a mana^ment technology for instruction* 
PSt, the j;jersondh/ed system of tnstritction^ often referred to as the "Keller Plan/' mscon<^(ved by Dr. fred S. 
KeUer in its most curretu form. A mmitier of variations exist even on the Stony Brook €:ampu$ whete it is being 
imii in Ed4H:dt{on, Physics, l^syrhology, and Biology^ to name a few departments. Like A T, the lecture is greatly 
deemphasi/ed as a rmsdium of information transfer, tn fact, it is used as a few^; the professor only gives his 
lectun? on a topic m wh«;h he is personally mt«»rested and atM?ut which he feels he has som^hing worthwhile to 
say. Students who attend only do so because they want to. 

Keller lists tJie following five essential features of PSf: (1) self-pacing; (2)( unrt-^asterv for ^vancement; 
13} the lecture as a f^atd; i4) written inter^tions t^tween student and teacher; and (5) the use of proctors for 
testing {oft*;n repeatedly), immudrate scoring, tutoring, and inter-personal interactic^. 

The University of Texa% . , Musim began with PSI in 1966 in Psychoio^cal Founv. ■ 'is of Stecondary 
Education. Now there are thirty court^es tau^t in fifteen departments in six colleges, and all without administra- 
tive coersion. This is not to sugc^st that there ar^ no snags or protdems. Stice and Hereford ghfe a bataiK:ed report 
and indica;te that they had problems and were led to seek funding f r€>m the outside* The Sloan Foundation^ of 
New York City, is now underwi^iting the program for two years- The funds are being used to ctevelop iof redesign) 
twelve PSi courses and to evaluate coursi; tiaia in an attempt to answer a set of questions. Both formative 
|"internar') and summanve (compai ative) research and evaluation studies are in fvocess. An important area of 
interest is feedback to instructors and proctors. The variety of cmir^ cm ^e one campus makes it possible to 
study PSI across courses, ctepartmental, and colle^ boundaries SO'^ to study ttus method itself. Even ba$\c prob- 
lems in educational researdi are being raised and studied since the methods l^at were developed for grouf^d and 
bounded learning < nviriX»ments do not ^ply to individiwliied and unboorKted environments. 

Also of interest is the study of student characteristics in relation to the effectiveness of PSI courses. This is 
a topic of interest to a number of the technologies tiecause it is so fundamental to the individualization of 
instruction. 

Recommendation 14: T>)at furxls be provided to sumnarize the research w individual differences so as to 
identify characteristics of students and working hypotheses regarding their interaction wfth conditions and varia* 
bles in learning environments. ^ 

The cost issues were not dodyud, and ttie ^>attern common to the technologies utilizing more hardware seems 
to tie presfitt with PSI as wi?il. *'CmX and time will he relatively high during the initia! offering of a PSI course but 
wMI drop m subMrc|uenl SiirnestiTS to a level comparable to that for more traditional forms of instruction." 

Havmt| develof>ed a course? an(i tho requtrod materials, cost benefits could result if it could be transferred to 
another mstitution. 

Recomnwndation 15: That hinds be made availahie to study the transfer problcnrs associated with PSI 
courses and othrr tectmolotjit'S as well. 
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Pf ofessor Duncan Hansen's paper is not only one of the most fact-packed but also one of the more contro- 
versial papers presented m the symposium. iSee Dr. Setdel's discussion.) He cites a number of mterestmg and 
r<!ttfvant research sfu<tfes conducted at FSU refatrng mainly to CMI. His position is ''Using a computer^naged 
ui&tructionaf (CMt) model to encompass computer assisted instruction {CAIKsimufation, adi^^tive testing, 
naturat language diaiogiKfs, media marta^ntent« scheduling, record Raping and evaluation, the potential of each 
of these components as training procedures L^omes most enhanced.** The studies he refers to were reported in 
technical reports to f imdtng sources. They are orgdnized into four oatagories: learner strategies, training 
strategies, vaticbtton strategies, and computer system strategies. 

Professw Hansen argues that the emphasis should be on a mana^ment model not on technology. For him 
this means tfiat the management of the entire enterprise for instruction, not Just the nawer t^hnotgical aspects. 
While this is certainly a valid pomt, the distinctkin between the management ^^ehrrolc^ and the delmry 
technology is interesting. However, it also is the case that toth are educational tedincdogies and botfi are necessary. 
Hansen's point is that the need now is for a com^uter-mana^ instructional <CWI^ moitel that encompasses CAI 
and a whole host of other functions. Like f^ofessor Lumsden, however from a different but related perspective. 
Professor Hansen argues for a strong and muttif^ted national research effort. "The primary research need is to 
extend ttie set of available indices reflecting training/' A "four strategy dieme approach" is deK?rited for tite work 
at Florida State Uiiiversity (FSU): learner strategic, training strategies, validation strategies, and computer systenni 
strategies. 

Learner-strategy research dealt with; rule learning; behavioral objecttve learning; memory; subjective organi- 
zation; anxiety and curiosity; and graphics. Several interesting findinf^ were reported. For example. "Results of 
research into the role of behavioral obj^tives m teaming and into st^ilective organizaticni have led investigators to 
feel that these are tess rewarding areas." Hansen also refK^rts that for rule learning, "1- Instructkm sftould present 
general instructional ob^cttws to inform the student as to what is expected of him. 2* Pr^ntation of rule Siate- 
ment can prevent the 'discovery' of an incorrect rule. 3. ft^esentation of sample test iterra gives the student a chance 
for practice and immediate feedback." Probably more controversial than other data were those r^rted relating to 
; the anxiety and curiosity research. 

Professor Hansen recommends several lines of research for the future in each of ^e four are^. One of the 
arca% he mentions that also was mentioned repeatedly in the symposium was re^arch on the processes of 
revision, 

Recommmidati<^ 16: That furuJs be provided to support the systematic study of the process and methodolc^y 
of materials revisiwi based upon empirical data. 

Recommendation 17: That funds be provided to conduct research on adapti^ and tailored testing, especially 
as it relates to student evaluation and to materials revision. 

Professor Brennan presented "A Model for the Use of Achievement Data and Time Data in an Instructional 
System." He considered both theoretical and practical issues involving the use of these two types of data in evaluating 
an instt uctional system. The two previous recommendations also could be appropriately made as inferences from 
Professor Brennan's paper. 

Professor Brennan presents a taxonomy of test data and time data for evaluation purposes, both students and 
instriictjon. His thesis is that evaluation is a fundamental part of curriculum development, not merely an appendage 
added on after the fact. It requires planning as well as technological resources, specially research design, reliable and 
valid instruments, and data. He presents an evaluation model which uses achievement and time data and is general- 
izable to many modes of instruction as well as to instructional systems, provided they have clearly stated inject ives 
while most of the literature currently available on evaluation treats the analysis of data using classical descriptive 
and inferential statistics. Brennan predicts that Bayesian and decision theoretic techniques will play an increasin^y 
important role in the evaluation of instructional systems. 
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Recotnmemtatffm 18: That funds be provided to cfevetc^ models and methocfe^ jncfiKling comfHiter software^ 
for the use of Bayesian statistics and decision theoretic stattstical techniques to the rangp of relevant problems in 
educational techmHogy. 

Or, Anastasio briefly reported on tf^ NSf funded evaluaticm project at the Educational Testing Service 
(ETSI which is just beginnit^. An interesting aspect of this project is that a separate a^ncy is faetng supported to 
evaluate these two CAI projects white they themselves ^re evohnng. This provides m c^portunity seldom enjoyed 
in education, namely, to (lermit the ^oup do»ng the devei(M:)ntent to receive continuous and ccmstructive input at a 
time vwhen it can be useful rather than after the fact* 

Recommemiation f9 .That fund i be made available to have independent evafuatton studies made of a 
variety of educational technologies <e.<f,, A T, Keller Plan, Vitteo systems) while they are being developed. 

Dr. Seidel, Human Resources R^seaHch Ckgdnii:atfon« rais^ nniny questions not only about the papers relat- 
ing to «ndividuali2^ and bounded environments tnit also about genera! issi^s. example^ he felt that 
discussion surrounding the concept of validity (wt^ther it was ccmcurrent, predictive or construct validity) to be 
interpreted differently by the various speakers. Secxmdiy, the concept of evatuaticm and the coiH^ept of techrrology 
^ means versus ends in themselves were unclear v^ith respect to education. The concept of evaluation also 
seemed to have different meaning for different people although nowhere explicitly stated. Finally^ the questions 
of mcentives for faculty members to use t^hnology, incentives for administrators to push the use of technoki^, 
and the guidetir^s for use of technolc^ in an educational environment alt seemed to be somewhat ambiguously 
toiK:heJ upc^ and nowherp clarified to anyone's satisfaction. 

''Nowhere did I find that we as a group ever came to grips with the definition of the concept of productivity 
....However, more than once I noted the pres^ce of cost/effectiveness as the paramount raison d'etre for the use 
of technology. Also, as noted atoi^, technolc^ w^ impiicitiy assume to be inextricably interwoven with state- 
ments of ends or educational purposes. (The pc^^er by Bess toughed on important issues of measurement but 
unfortunately added to ttie means^end ctmfusion,)" 

'"Perhaps the tatter is the place to start to try to clarify my own position with respect to the ambiguities as 
1 saw them in that conference. From our work at HumRRO. it has become quite clear to us that there are multi- 
ple purposes for using ttie computer as a r^w technology in educaticm and that in only orie of ^ese purposes, 
that is, cost/effectiveness purpose, is it apprc^riate to talk about fixed objectives and fixed economiis for making 
evaluations of alternative means of instruction. (We are including this discussion in our book.) Gkit tfiere are other 
purposes for which computers are also us«l (as I tried to indicate at the conferen^): Purpc^ such es computing 
literacy « enhancing the curricula for a cfisciptine, providing con^ting c^porninities as a tcK>l and finally, overall 
educa 'ional reform using the computer. The concepts of validity, methodology for evaluation, as well as criteria of 
evaluation all different dependent upon whi<ii of the above mentioned purposes are being prescribed. Dr. Filep 
made a strong case for the cost/eff^ive use of technology in education. He noted that $29.9 billion was spent 
for hi^er education in 1971 and that in 1981 the expenditures are likely to jump to S46 billion. Yet as he 
indicated there will be the same number of learners. Therefore, in order to cut costs, educaticKi^ technology can 
haw a very imfwtant role to play. Note, however, that this approach, this premise, or this purpose, drav/s one 
into a closed box of evaluation in that it ^umes objectives do not change and that economics do not change. 
Therefore, the whole concept of computers or other technology providing a "value added" capability cannot be 
^Kidressed, Yet education can viewed as an open-ended process ie^ing to new obj^tives and having a general 
transfer value for the individual and for society." 

"The other purposes noted above become more siltent when (mic asks the question, 'Is the current educational 
model appropriate?'. The issues arc^e in one presentation discussing the concept of the PSt model where the 
mdividual makmg the presentation felt that the old exams are not appropriate despite an orientation towards the use 
of behavioral objectives. His question was how do you compare the results of the PSI program to the so-called 
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conventional mode of presentation. Mv answ^ to this is quite simple: You cannot gain consensus on a criterion 
for measurement until you can agree on the objectives. Moreover, if new objectives are proposed, then the evalua- 
tton has to Ik? made in terms of fa) the acceptance tn not of the new objectives, and if they are accepted, (bj the 
effectiveness or viabiitty of the technology towards rrweting those obiectives. But in no way is it appitH^iate to 
make a comparison of cost/effectiveness between old and new since you have added something to the 
instructional repertoire which was not prc^nit previously . In this case it generally means that you have enhanced 
the curricula of the discipline/' 

"The methodology and the criteria for evaluation tecome even more difficult to conceive of in dassic 
terms wheri the goal of using technology (my interest teing in use of the computcnrl is towards the process or 
experience of using the comptiter's technotc^ical capability as a tool for student growth. This is opposed to a 
purpose which focusses on a product as a result of the use of the technology, that product temg mastery of some 
specified set of behavioral objectiveSv And when one stretches the purfK>se even further towards the affective 
<tomain sud» as creating a more positive attitude toward, and awarene^ of, the valu^ of technology and learning 
in ^neral, classic psychometrics are In no way relevant nor are mastery n^thoife relevant for ev.«.^i^ting the worth 
of tlie technology for education. Humanistic jud^ents of acceptance and long-term, broad-ranging effects cm 
the total person become the relevant criteria. Here is where t see substantial value for Bess' measurement 
interest (broad suddl effects).^ 

*'|n short, the purposes of education and the purposes for the use of the technology within a given role of 
education or educational model must be clearly stipulated in (H^der tfrat one can speak without ambiguity of the 
value of technology for education. I^oductivity similarly must be defined in terms of the purpose for which 
education is being provided as opposed to a simptisiic impKcation from our industrial society {see, for example, 
Hagerty^ Sducstion^ /7«s»arr/^^ October, 1972) towards inc^asing the number of units of output f^t unit 
time spent in the production process." 

"Lastly, I was very pleased to hear the group address the fxoblem of modds for implementaticKi of new 
technologies in terms of the organizational structures which mi^t provide the most viable entities in order to 
make the development of such technolc^y a feasible endeavor within educatic^. Serious qimtions have to be 
considered concerning the role of a sin^ orthodox institution in terms of its viability as the tiasis for imple- 
menting technology on any continuing basis. This is especially true for the devetopmmt, dissemit^ticxi and use 
of computer-b^d curricula. Development complexities, different instructional orientation, specialized 
personnel requirements, and unique financial Uirdens alt require m innovative approach. The Discipline Center 
approach descrit)ed by Art Luehrman is one interesting type of nwidel which is worthy of further elaboration 
and consideration by the federal government and perhaps by regional organizations* For example, as we dis- 
cussed at one evening session, the state university system in New York mi^t well designate curriculum develop- 
ment and evaluation to be carried out at one center and that center would do minimal amounts of operational 
teaching. The responsibility for actual delivery and use would occur at other institutimis within the state 
system, etc." 

Dr. Seidei's points are important and underscore several of the previously made recommendations. His 
comments relating to an empirical study reported in Professor Han^n's paper, however, do sug^st a new 
recommendation. 

Recommendation 20: That funils be provided to conduct research on the relative effectiveness of the fol 
lowing four CAI strategies; adaptive model; remediation for-all, learner-choice; and no-remediation. 

The reported data from Rivers study, while interesting and provocative, were not definitive. Hansen's inter- 
pretation was chahen^^d, or, at least, qualified by Seidel It is significant enough to justify recommendation 20. 
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f^rsonirtized. Open teaming Enytrmments 

Oeati BaklwiTi reports on a pu-gram usiny videotapes in Engineering Education n<»v in operation at Colorado 
State Umversity, Seventeen t^itjinetfiii^coHe^s tiiive offered these proytams over the past nine yean^. This use of 
ITV systems is a significant f r^^tion of the U,S. hii#ief ediKnation experience m providing ^^uate instruction, for 
credit, at the student's place of empfoyment. The SURGt pro-am at Colorado State University is one of the 
largest and ddest. In surveying the on-going learning systems. Dean Saldwin reports that his data ".♦*,arB crude^ 
orcter-of-magniffade numbers../' The first national survey of these programs is now underway under the Chairman- 
ship of Albert J. Morr s. The study is called ''The Cost-Effectiveness of Conthiuing Engineer na Studies by 
Television/* What is needed ts a comf^rabte study of applications <n other areas such as urn r* . duate instruction. 
The results could provide guidance to institutions wanting to do simitar things. 

Recwnmem/ation 21: That a naticmai survey be conducted to determine the nature, % u» cost, and bene- 
fits of video-based prc^rams in different areas of Education, both professional and nonpros -lonal. 

It is clear that video technology in hi^er education is becoming a %x\OfP and more hiteresting i«ay of relating 
to the existing needs for graduate instruction in engineering. But, Dean Baldwin also reported on the use of video 
instriATtion for undergraduates on campus, including a modified Keller plan and a career ^idance program on 
dormitory CATV. Their extensions of video technology at Colorado State University hsB interesting implications, 
but the extension is not limited to undergraduates. Video technology also is beirtg used in Continuitrg Education 
to meet the needs of older (ovw 40) engineers, to the career guidance of ycHing students and the "open ieamirtg 
concept/' Many of the s^nie materials can tK? used to meet these different needs. 

Since SURGE, iot example, has general^ tapes and associated course materials for use in off-campus centers, 
it became a relatively simple matter to use them for other groups as wdl. Dean Baldwin reported a successful new 
use in summer school. In f^t, erne faculty memfcter combined their use for the format of a Keller plan to make tfie 
teaming environment more individualized. In additicm, a low^^livision eiecth^ course was offered during the 
re^iar semester, in the modified Keller n^thod. Of interest is the ixi that ttre video t<^, in this case, was prepared 
by a professor who is on another faculty. This ts only <»ie way of achieving a shared distritnition of materials, Tfie 
iitseralization of attitudes about sharing materials of this kind needs to be examined to determine a variety of ways 
of making materials more acceptable throu^out ttre community of professors who are the consumers. Unlike other 
economic systems, the ultimate consum«>r, the student, dc^ not now select ^ materials he pays for and uses. 

Dean Baldwin feels that "..,.oW budget prentices... preclude a fair hearimi of tedinically twsed instructional 
cations." The most serious problem. In his view as a middte-tevet mana^r, is "the total l^k of conskieration which 
is given to capital outlay amortization in budget preparation..." He also points out that "ti^t budgets" are 
wedded to fixed labor costs and perpetuate the spiraling costs of a labor intensive system." 

Recowm&idation 22: That a study be conducted to determine alternative budget models and ^sociated 
materials that will be useful to universities and collets wanting to adopt the new technologies. 

Dean Zigerall points up the need for an reasonableness of evaluation perspective, the use in evaluation of 
what the project planners set as their goals and objectives. Sef>arate from this is the question of y\rhether they 
chose wisely and well in relation to the larger society and its needs. These are separate issues. We tend to overlook 
the former or confuse it with the latter. The Chicago TV College, an eighteen-year old effort, is an extension of 
the City Colleges of Chicago, a public two-year community coW^, It is interesting thjt the TV College is now a 
part of a unit— the Learning Resources Laboratory— which will pfoysi^e leamirrg specialists and a variety of 
instructional materials for the seven colle^ making up the City Colfeges of Chicago. This fact should be viewed 
in the light of the points previously made about the sociological factors and the contr^xt in which technology must 
yow and develop and the recommendation made atKHit Education Experimental Station needs. It woul be 
interesting to note what the critical factors were that led to this shift. Also of interest would be the institutional 
administrative plan since these kinds of information could 1;^ useful to othi*r groups contemplating a similar 
development. 
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T|)e Chicago TV Colleye also rtipres<»nts a multifaceted f;K:$<itY. It not only fwoduces mater tals for tetevision 
iiroadcabt, it also pi oditces vitieocas&etfts tor ft^ Chioigo area. The Miter aitows for th^ greater fi^rso^aiization 
of imtriicf ion. Plans ore b^uu} nidtlt? to use four channeh of an ITFS system ds well. The audience potenti^ is 
vif itialty unknown, hut a$^unttiil to iio ''Hiexhaustilite/' 

Recommendation 23: That funtis he proviiteif to study the audif^nce needs and desires in major metropolitan 
atfids and to study the afuinnt of the Chica^ TV Coiie^» to see what career and long-range imf^t their TV 
instruction has had. 

Other prevsously made recommendations are afso supported by Dean Zi^HrBtf's pd|X>r, especiafly the recom* 
mendiitioii that the problems of transferring technology to other mstituttons b«» studied. It is (^rticulariy interesting 
m this regard to note that a coitsistent 40 percent of TV CoUege-credit students have regularly expressed a desire 
to I)ec6me tethers. As a result, a number of nHevant courses exist for teachers and could be transferred to other 
locations. 

Recommendation 24: That funcis made available to U\ conduct a matket analysis for the teacher 
education TV courses to determme the demand for them in other locations, and <b) to ctetermine the problems 
and cost of transfwiny them. 

The need is for an analysis of the problems of intermstitutional transfer. Uruloubtedly this is rooted in 
attittides of faculty, but the problem is sufficiently serious and promises cost benefits that need to be taken 
advantage of. 

Brian Lewis presents a fact-packed p^>er; a comprehensive account of the British Oj^n Uniiwrsity's main 
teaching arrangements. Also includeti are jdata bastnl upon their experience since 1969, yvhen it was established 
by Royal Charter. Teaching, however, beg^n in 1971, but the two years were fitted with ^ttvities that were new 
and unprecedented approaches to problems in higher education. This is the first ".. .full-scale home-bds^, multi- 
media system of higher education." ft ptuvides <H>Portunities at both the undergraduate and post^audate levels 
to all who are precluded from realizing their educational aims in an existing institution, in spite of these limita- 
tions in time and breatlth of scope, twenty thousand students actually paid llieir fees and started in Jmuary, 
1971. As many as &0 percent stayed in, arnJ 75 percent passed their examinations. 1972 saw a ref^tition of this, 
and the current student population is approximately forty thousartd students. 

The mam student location is the home, but four te^hing resources are available: postal packages of home- 
study materials; television and radio; local study centers; and short residential summer schools. The first of these 
constitutes the "core" content of the courses. To get a itegree, a student must suixessfufly ccwnplete six to eight 
cours*?s of study at a rate of not more than two courses per year. Each coiirse requires twelve hours of hon>e 
study per week for nine months. The students are adults over twenty one. but plans to admit younger students 
exist. 

The courses g<ve students many things to do. and each study f>ackdge is self-contained and sustains ^tivity 
for four to six weeks. Radio and television supplement and complement the core materials. Students return the 
homework within specific time limits. 

The constraints experienced by the "faculty" are more in the developnwnt of materials than in their 
delivery Assigned work is partially processed by computers. There are tutor-marked assi^ments as well. For 
each homework assignment the computer generates a detaileii statement of how well the student has done. 
Progress reports are made to regional offices and to students. Student opinion is routinely solicit«l, and question 
naires are regularly sent to representative samples of students. These kinds of feedback are pr<K:e5sed. O.U, has 
its own Institute of Educational Technology which, among other activities, has primary responsibdity for guttling 
and informing the process of self-improvement. 
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Dr. Lewis reported In usefui litetaH the prc^tems and processes in course de^opment. Umierestimation of 
the task and iimited resources have bunfened tile staff members, but '%,.the ii^odwtion of quality cmirses is now 
merely difficult, rather than imfKsnifaie/' It takes two to three ^meks to produce an actec^te first draft, which 
then needs to be tried out or tested with a sample of ap^c^iate students. Then revision foliows, and the new 
draft is tested-a test revise retest cyde-until the author is satisfied. Each new cycle adds three to fmn weeks, at 
least. In addition, the author must devise apprcH^r^ite homework. In doing this, he must specify exactly the plan 
for markiny to be used by the computer, Patterns of staff differentiation in funi^ion have emer^ and are 
described. 

The use of "self-contained blocks** of foir toet^t units e«;h, authored separately, has resulted in an 
effective strategy for producing courses that run thirty-six weeks' duraticm. Some orte4ialf raui^ or one-third 
courses also are being tteveloiJ^d. 

ftetomfimutatiaa 25: That funds be f»^ovicfed to develop, out of tfw O.U. exf^rience, for example, recpiire-^ 
ments and specifications for as yet UfKieveif^jed alcte to course dev^opment,soas to increase the productivity 
of the course development process, 

ftecommendatitm 26: That funds be provided to plan and dev^op a i»t of courses to t» used by an c^n 
university network. 

This recommendation was made by Chancellor Upson. He included in bis paper a detailed prcqsosal for 
such an effort. His bench mark is 3(X> cotiN^, and he estimates a $60 million per year a9St for a five-year period. 
A cooperati^ effort is envis med for: (a) production of course materials; (b) systems of student acx»ss; <c} 
proficiency tests; and (d) evaluation of the system. 

Professor Klare reported on "readattlity^ studio of cxirn^ondenc^ materials like the "core materials'* 
used in the O.U. At least twenty-nine compu^ pro|^ams curr^tly exist for af^yjng read^lity formulas to 
large bodies of text. The studies indicated wome interesting relationships betw^n readmg difficuky and drt^ 
out. These studies of USAFI {United States Armed Forces Institute) data are from thirty courses. A rank-order 
correlation of .87 was fourtd between the readability scores and the probability that students would send in ail 
of their lessons. Clearly read^ihty is related to persistence. 

ftecommwdation 27: That funds be provided to conduct experimental stiKiies of the relationship of 
comprehension and persistence to readability by using materials desigr>ed to m^t a single set of (^jectives, but 
written at different levels of readat^lity. 

Recommendation 28: That funds be provlcted to develop inter^tive versions of readability programs to aid 
authors in developing materials for tar^ted stuctent users. 

Professor Yelon addressed himself to the pa^s by Baldwin and Lewis with respect to m^urement aiKi 
evaluation. In his opinion {^an Baldwin did not look at all important producth^ity measures; he did not mention 
the application (transfer) of learning or the achieving of more objectives. Lewis omitted stiKlent learning in his 
summary of the O.U. experience. 

Recommendation 29: That funds be provided to develop a comprehensive list and appropriate guides for 
collecting productivity data in evaluation of educational technology procrams. 

Yefon also raised one of tf» sensitive questicK>$, procedures for a^^lying social science to development, 
delivery, ami evaluation. Probably the single most promising and cost-effective solution to the problems of 
educational technology is a good social science theory and techniques for applying it. Therefore our final 
recommendation based on this symposium is the following: 
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Recommendation 30: That funch be provided to develop social science ami behavioral theories that relate 
to the probtems of educational, and especiaHy, mstructronal tedinoloav* 

» 
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